2022-02-16T23:41:11.771 INFO:root:teuthology version: 1.1.0-c6442aa9 2022-02-16T23:41:11.773 INFO:teuthology.results:subset: '1/5000' 2022-02-16T23:41:11.773 INFO:teuthology.results:seed: '570' 2022-02-17T14:23:07.434 INFO:root:teuthology version: 1.1.0-c6442aa9 2022-02-17T14:23:07.491 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-02-17T21:00:41.174 INFO:teuthology.results:Tests finished! gathering results... 2022-02-17T21:00:59.704 INFO:scrape:Found 206 jobs 2022-02-17T21:01:00.494 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-10700-g308702d2 (308702d20411f080c7f89b070f211bcd03e80097) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12c20) [0x7fd7a6681c20] , 2: gsignal() 2022-02-17T21:01:00.495 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.313 INFO:teuthology.orchestra.run.smithi106.stdout: "duration": 4.7528000000000003e-05 , 2022-02-17T10:11:56.313 INFO:teuthology.orchestra.run.smithi106.stdout: }, , 2022-02-17T10:11:56.313 INFO:teuthology.orchestra.run.smithi106.stdout: { 2022-02-17T21:01:00.496 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.337 INFO:teuthology.orchestra.run.smithi106.stdout: "time": "2022-02-17T10:10:43.155824+0000", , 2022-02-17T10:11:56.337 INFO:teuthology.orchestra.run.smithi106.stdout: "duration": 9.2299999999999997e-06 , 2022-02-17T10:11:56.338 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2022-02-17T21:01:00.497 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.360 INFO:teuthology.orchestra.run.smithi106.stdout: "event": "initiated", , 2022-02-17T10:11:56.360 INFO:teuthology.orchestra.run.smithi106.stdout: "time": "2022-02-17T10:10:43.157887+0000", , 2022-02-17T10:11:56.360 INFO:teuthology.orchestra.run.smithi106.stdout: "duration": 0 2022-02-17T21:01:00.497 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.383 INFO:teuthology.orchestra.run.smithi106.stdout: "event": "done", , 2022-02-17T10:11:56.383 INFO:teuthology.orchestra.run.smithi106.stdout: "time": "2022-02-17T10:10:43.225901+0000", , 2022-02-17T10:11:56.383 INFO:teuthology.orchestra.run.smithi106.stdout: "duration": 9.6749999999999997e-06 2022-02-17T21:01:00.498 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.406 INFO:teuthology.orchestra.run.smithi106.stdout: "client": "client.4448", , 2022-02-17T10:11:56.406 INFO:teuthology.orchestra.run.smithi106.stdout: "client_addr": "v1:172.21.15.104:0/2107151390", , 2022-02-17T10:11:56.406 INFO:teuthology.orchestra.run.smithi106.stdout: "tid": 89791 2022-02-17T21:01:00.498 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.429 INFO:teuthology.orchestra.run.smithi106.stdout: { , 2022-02-17T10:11:56.429 INFO:teuthology.orchestra.run.smithi106.stdout: "event": "all_read", , 2022-02-17T10:11:56.429 INFO:teuthology.orchestra.run.smithi106.stdout: "time": "2022-02-17T10:11:30.043787+0000", 2022-02-17T21:01:00.499 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.451 INFO:teuthology.orchestra.run.smithi106.stdout: "initiated_at": "2022-02-17T10:11:35.020763+0000", , 2022-02-17T10:11:56.451 INFO:teuthology.orchestra.run.smithi106.stdout: "age": 21.276292466000001, , 2022-02-17T10:11:56.452 INFO:teuthology.orchestra.run.smithi106.stdout: "duration": 0.050506561999999998, 2022-02-17T21:01:00.499 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.474 INFO:teuthology.orchestra.run.smithi106.stdout: }, , 2022-02-17T10:11:56.474 INFO:teuthology.orchestra.run.smithi106.stdout: { , 2022-02-17T10:11:56.474 INFO:teuthology.orchestra.run.smithi106.stdout: "event": "header_read", 2022-02-17T21:01:00.500 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.497 INFO:teuthology.orchestra.run.smithi106.stdout: } , 2022-02-17T10:11:56.497 INFO:teuthology.orchestra.run.smithi106.stdout: ] , 2022-02-17T10:11:56.498 INFO:teuthology.orchestra.run.smithi106.stdout: } 2022-02-17T21:01:00.501 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.520 INFO:teuthology.orchestra.run.smithi106.stdout: "duration": 4.4000000000000002e-06 , 2022-02-17T10:11:56.520 INFO:teuthology.orchestra.run.smithi106.stdout: }, , 2022-02-17T10:11:56.520 INFO:teuthology.orchestra.run.smithi106.stdout: { 2022-02-17T21:01:00.501 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.542 INFO:teuthology.orchestra.run.smithi106.stdout: "duration": 4.2734000000000001e-05 , 2022-02-17T10:11:56.543 INFO:teuthology.orchestra.run.smithi106.stdout: }, , 2022-02-17T10:11:56.543 INFO:teuthology.orchestra.run.smithi106.stdout: { 2022-02-17T21:01:00.502 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.565 INFO:teuthology.orchestra.run.smithi106.stdout: "type_data": { , 2022-02-17T10:11:56.565 INFO:teuthology.orchestra.run.smithi106.stdout: "flag_point": "commit sent; apply or cleanup", , 2022-02-17T10:11:56.566 INFO:teuthology.orchestra.run.smithi106.stdout: "events": [ 2022-02-17T21:01:00.502 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.588 INFO:teuthology.orchestra.run.smithi106.stdout: "time": "2022-02-17T10:11:44.239501+0000", , 2022-02-17T10:11:56.588 INFO:teuthology.orchestra.run.smithi106.stdout: "duration": 6.6820000000000001e-05 , 2022-02-17T10:11:56.589 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2022-02-17T21:01:00.503 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:56.615+0000 7f16553db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28148) UID: 0 , 2022-02-17T10:11:56.716+0000 7f23e2115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 28186) UID: 0 , 2022-02-17T10:11:56.789 INFO:teuthology.orchestra.run.smithi104.stdout:osd.0: {} 2022-02-17T21:01:00.504 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:11:58.331+0000 7f7a8b1ce700 -1 osd.0 63 *** Got signal Terminated *** , 2022-02-17T10:11:58.331+0000 7f7a8b1ce700 -1 osd.0 63 *** Immediate shutdown (osd_fast_shutdown=true) *** , 2022-02-17T10:11:58.335+0000 7f247cf2b700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28150) UID: 0 2022-02-17T21:01:00.504 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.505 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.505 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.506 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:00.528 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.507 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:01.062 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.507 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.508 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.508 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.509 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.510 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.510 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.511 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:12:04.874 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.512 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:05.380 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.512 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.513 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.513 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.514 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.515 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.515 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.516 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:09.017 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.516 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:09.538 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.517 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.517 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.518 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.519 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.519 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.520 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:12:12.619 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.520 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:13.185 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.521 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:13.698 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.522 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.522 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.523 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.523 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.524 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.524 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.525 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:17.324 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.526 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:17.856 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.526 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.527 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.527 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.528 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.529 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.529 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.530 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:21.482 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.530 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:22.015 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.531 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.532 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.532 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.533 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.533 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.534 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.534 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:25.609 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.535 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:26.172 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.536 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.536 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.537 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.538 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.538 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.539 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.539 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:30.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.540 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:30.589 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.540 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.541 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.542 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.542 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.543 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.543 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.544 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:34.176 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.544 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:34.744 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:34.745 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:00.545 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.546 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.546 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.547 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.547 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.548 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.549 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:38.318 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.549 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:38.869 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:38.870 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:00.550 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.550 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.551 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.551 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.552 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.553 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.553 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.554 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:43.017 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:43.018 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:00.554 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:43.567 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:43.568 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:12:43.574 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.555 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.556 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.556 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.557 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.557 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.558 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.559 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:47.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:47.156 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:00.559 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:47.719 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:47.720 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:12:47.728 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.560 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.560 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.561 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.562 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.562 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.563 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.563 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:51.296 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:51.297 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:00.564 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:51.848 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:51.849 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:12:51.886 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.564 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.565 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.566 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.566 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.567 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.567 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:12:55.137 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.568 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:55.688 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:55.689 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:00.569 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:12:56.251 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:56.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:12:56.253 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.569 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.570 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.570 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.571 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.572 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.572 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.573 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:12:59.828 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:12:59.829 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2022-02-17T21:01:00.573 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:00.378 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:00.379 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:13:00.407 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.574 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.575 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.575 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.576 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.576 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.577 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.577 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:13:03.960 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:03.961 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2022-02-17T21:01:00.578 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:04.512 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:04.513 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:13:04.562 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.579 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:05.068 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:13:05.069 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.579 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.580 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.580 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.581 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.582 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.582 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.583 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:08.662 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:08.663 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:13:08.718 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.583 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:09.214 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:13:09.225 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.584 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.585 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.585 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.586 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.586 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.587 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.588 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:12.792 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:12.793 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:13:12.876 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.588 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:13.358 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:13:13.384 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.589 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.589 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.590 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.591 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.591 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.592 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.592 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:13:17.031 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.593 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:17.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:13:17.538 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.594 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.594 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.595 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.595 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.596 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.597 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.597 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.598 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:21.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:13:21.695 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.598 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:22.203 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.599 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.600 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.600 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.601 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.601 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.602 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.603 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:25.988 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:13:26.076 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.603 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:26.583 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.604 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.605 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.605 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.606 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.606 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.607 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.608 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:30.138 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:13:30.237 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.608 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:30.745 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.609 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.609 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.610 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.611 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.611 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.612 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.613 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:13:34.369 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.613 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:34.903 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.614 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.614 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.615 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.616 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.616 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.617 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.617 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:13:38.507 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.618 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:13:39.060 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.618 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.619 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.620 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.620 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.621 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.621 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.622 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:13:42.645 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.623 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:13:43.209 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:43.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:00.623 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.624 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.624 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.625 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.626 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.626 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.627 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:13:46.791 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.627 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:13:47.341 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:47.342 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:00.628 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.629 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.629 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.630 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.630 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.631 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.632 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.632 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:13:51.863 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:51.864 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:00.633 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:52.413 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:52.414 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:13:52.450 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.633 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.634 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.635 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.635 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.636 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.636 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.637 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.638 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:56.556 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:13:56.557 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:13:56.610 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.638 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:13:57.107 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:13:57.117 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.639 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_ceph-c_308702d20411f080c7f89b070f211bcd03e80097/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd , return self.run_cluster_cmd(**kwargs).stdout.getvalue() , File "/home/teuthworker/src/git.ceph.com_ceph-c_308702d20411f080c7f89b070f211bcd03e80097/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd 2022-02-17T21:01:00.639 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.640 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.641 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.641 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.642 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.642 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:00.503 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.643 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:01.032 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:01.070 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.644 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:01.580 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.644 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.645 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.645 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.646 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.647 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.647 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.648 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:05.196 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.648 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:05.736 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.649 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.649 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.650 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.651 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.651 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.652 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.652 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:09.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.653 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:09.881 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:14:09.882 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2022-02-17T21:01:00.654 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.654 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.655 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.656 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.656 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.657 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.657 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:13.455 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.658 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:14.019 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:14:14.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:00.659 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.659 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.660 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.660 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.661 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.662 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.663 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:14:17.604 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.664 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:18.154 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:14:18.155 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:00.665 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:18.718 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.665 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.666 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.666 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.667 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.668 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.668 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.669 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:22.731 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:14:22.731 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:00.669 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:23.280 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:14:23.281 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:14:23.325 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.670 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.671 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.671 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.672 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.672 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.673 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.674 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.674 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:27.423 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:14:27.424 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:14:27.481 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.675 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:27.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:14:27.987 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.675 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.676 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.676 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.677 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.678 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.678 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.679 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:31.543 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:14:31.544 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:14:31.636 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.679 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:32.111 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:14:32.144 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.680 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.681 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.681 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.682 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.682 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.683 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.684 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:14:35.795 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.684 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:36.239 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:14:36.302 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.685 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:36.810 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.685 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.686 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.686 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.687 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.688 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.688 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.689 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:40.385 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:14:40.462 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.689 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:40.970 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.690 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.691 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.691 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.692 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.692 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.693 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.694 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:44.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:14:44.623 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.694 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:45.131 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.695 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.695 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.696 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.696 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.697 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.698 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.698 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:48.899 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.699 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:49.412 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.700 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.700 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.701 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.701 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.702 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.703 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.703 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:53.030 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.704 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:14:53.574 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.704 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.705 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.706 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.706 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.707 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.707 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.708 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:57.175 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.709 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:14:57.729 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:14:57.730 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:00.709 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.710 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.710 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.711 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.712 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.712 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.713 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:01.303 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.713 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:01.871 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:01.872 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:00.714 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.714 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.715 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.716 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.716 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.717 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.717 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:15:05.449 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.718 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:05.999 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:06.000 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:00.719 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:06.564 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.719 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.720 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.720 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.721 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.721 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.722 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.723 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:10.129 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:10.130 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2022-02-17T21:01:00.723 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:10.680 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:10.681 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:15:10.727 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.724 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.724 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.725 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.726 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.726 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.727 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.727 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.728 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:14.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:14.984 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:15:15.057 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.729 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:15.535 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:15:15.562 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.729 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.730 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.730 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.731 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.731 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.732 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.733 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:19.122 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:19.124 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:15:19.213 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.733 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:19.679 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:15:19.719 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.734 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.734 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.735 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.736 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.736 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.737 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.737 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.738 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:23.824 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:15:23.873 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.738 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:24.379 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.739 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.740 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.740 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.741 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.741 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.742 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.743 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:27.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:15:28.029 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.743 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:28.537 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.744 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.745 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.745 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.746 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.746 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.747 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.747 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:15:32.193 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.748 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:32.700 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.749 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.749 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.750 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.750 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.751 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.752 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.752 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:36.317 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.753 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:36.856 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.753 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.754 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.754 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.755 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.756 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.756 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.757 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:40.454 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.757 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:41.004 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:41.005 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:00.758 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.759 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.759 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.760 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.760 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.761 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.762 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:44.726 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.762 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:45.294 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:45.294 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:00.763 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.763 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.764 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.764 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.765 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.766 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.766 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:48.866 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.767 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:49.418 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:49.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:00.768 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.768 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.769 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.769 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.770 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.771 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.771 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.772 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:53.555 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:53.556 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:00.772 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:54.108 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:54.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:15:54.110 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.773 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.774 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.774 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.775 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.775 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.776 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.777 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:15:57.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:57.683 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:00.777 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:15:58.248 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:15:58.249 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:15:58.267 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.778 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.778 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.779 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.780 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.780 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.781 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.781 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.782 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:02.381 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.783 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:02.929 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.783 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:03.436 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.784 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.784 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.785 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.786 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.786 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.787 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.787 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:07.085 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.788 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:07.595 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.789 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.789 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.790 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.790 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.791 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.792 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.792 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:11.225 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.793 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:11.755 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.793 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.794 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.795 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.795 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.796 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.796 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:16:15.000 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.797 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:15.554 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.798 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:16.115 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.798 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.799 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.799 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.800 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.801 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.801 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.802 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:19.704 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.802 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:20.256 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:16:20.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:00.803 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.804 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.804 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.805 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.805 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.806 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.807 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:23.831 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.807 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:24.397 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:16:24.398 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:00.808 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.808 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.809 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.810 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.810 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.811 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.811 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:27.980 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.812 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:28.531 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:16:28.531 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:00.813 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.813 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.814 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.814 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.815 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.815 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.816 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.817 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:32.670 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:16:32.671 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:00.817 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:33.223 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:16:33.224 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:16:33.246 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.818 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.819 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.819 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.820 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.820 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.821 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.821 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:36.797 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:16:36.798 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:00.822 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:37.371 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:16:37.372 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:16:37.404 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.823 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.823 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.824 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.825 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.825 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.826 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.826 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.827 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:16:41.704 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.828 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:42.171 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:16:42.211 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.828 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.829 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.829 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.830 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.830 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.831 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.832 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.832 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:46.297 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:16:46.367 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.833 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:46.874 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.833 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.834 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.835 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.835 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.836 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.836 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.837 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:50.434 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:16:50.525 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.838 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:51.032 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.838 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.839 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.839 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.840 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.841 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.841 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.842 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:54.679 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.842 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:55.192 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.843 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.844 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.844 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.845 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.845 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.846 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.847 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:16:58.823 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.847 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:16:59.350 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.848 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.848 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.849 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.849 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.850 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.851 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.851 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:02.970 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.852 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:03.503 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.852 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.853 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.854 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.854 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.855 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.855 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.856 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.857 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:07.760 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:07.761 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:00.857 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.858 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.858 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.859 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.860 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.860 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.861 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.861 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:11.882 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:11.883 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2022-02-17T21:01:00.862 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:12.450 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:12.451 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:17:12.466 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.863 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.863 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.864 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.864 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.865 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.866 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.866 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:16.032 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:16.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:00.867 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:16.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:16.586 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:17:16.623 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.867 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.868 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.869 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.869 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.870 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.870 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.871 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.871 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:20.716 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:20.717 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:17:20.782 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.872 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:21.271 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:17:21.290 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.873 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.873 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.874 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.874 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.875 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.876 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.876 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:17:24.944 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.877 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:25.401 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:17:25.452 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.877 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:25.960 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.878 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.879 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.879 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.880 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.880 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.881 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.881 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:29.547 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:17:29.611 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.882 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:30.117 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.883 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.883 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.884 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.885 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.885 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.886 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.886 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:33.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:17:33.770 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.887 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:34.279 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.888 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.888 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.889 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.889 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.890 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.890 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.891 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:17:38.145 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.891 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:38.651 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.892 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.893 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.893 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.894 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.895 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.895 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.896 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:42.278 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.896 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:42.810 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.897 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.897 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.898 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.899 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.899 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.900 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.901 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:46.409 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.901 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:46.967 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.902 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.903 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.903 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.904 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.904 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.905 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.906 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:50.557 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.906 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:51.106 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:51.106 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:00.907 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.907 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.908 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.909 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.909 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.910 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.910 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:54.684 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.911 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:55.261 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:55.263 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:00.912 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.912 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.913 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.914 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.914 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.915 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.915 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.916 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:17:59.387 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:59.387 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:00.916 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:17:59.955 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:17:59.956 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:17:59.957 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.917 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.918 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.918 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.919 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.919 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.920 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.921 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.921 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:04.079 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.922 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:04.625 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.922 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:05.132 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.923 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.924 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.924 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.925 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.925 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.926 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.927 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:08.882 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.927 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:09.454 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:09.455 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:00.928 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.928 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.929 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.929 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.930 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.931 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.931 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:13.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.932 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:13.586 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:13.587 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:00.932 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.933 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.934 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.934 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.935 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.935 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.936 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.937 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:17.721 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:17.722 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:00.937 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:18.273 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:18.273 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:18:18.296 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.938 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.938 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.939 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.940 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.940 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.941 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.941 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:21.855 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:21.856 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:00.942 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:22.419 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:22.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:18:22.450 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.943 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.943 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.944 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.945 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.945 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.946 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.947 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.947 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:26.536 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:26.537 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:18:26.608 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.948 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:27.102 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:18:27.115 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.948 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.949 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.950 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.950 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.951 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.951 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.952 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:30.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:30.683 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:18:30.767 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.953 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:31.234 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:18:31.275 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.953 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.954 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.955 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.955 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.956 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.956 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.957 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:34.834 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:34.836 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:18:34.923 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.958 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:35.389 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:18:35.430 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.958 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.959 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.959 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.960 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.961 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.961 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.962 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.963 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:39.751 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:18:39.813 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.963 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:40.319 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.964 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.964 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.965 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.966 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.966 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:00.967 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.967 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:43.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:18:43.968 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:00.968 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:44.474 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.969 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:00.969 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.970 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:00.970 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.971 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.972 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.972 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:48.119 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.973 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:48.627 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.973 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.974 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.975 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.975 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.976 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.976 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:00.977 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:52.256 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.978 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:18:52.791 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:00.978 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.979 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:00.979 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.980 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:00.981 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.981 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.982 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:56.399 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.982 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:18:56.948 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:18:56.948 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:00.983 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.984 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.984 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.985 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.985 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.986 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.987 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:00.527 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.987 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:01.094 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:01.095 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:00.988 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.988 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.989 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:00.990 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.990 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:00.991 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.991 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:04.663 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:00.992 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:05.213 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:05.214 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:00.993 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:00.993 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.994 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.994 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:00.995 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:00.996 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:00.996 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:00.997 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:09.506 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:09.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:00.997 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:10.058 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:10.059 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:19:10.064 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:00.998 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:00.999 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:00.999 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.000 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.000 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.001 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.002 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:13.630 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:13.631 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:01.002 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:14.193 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:14.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:19:14.222 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.003 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.003 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.004 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.005 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.005 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.006 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.006 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:19:17.771 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:17.772 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:01.007 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:18.323 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:18.324 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:19:18.379 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.008 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.008 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.009 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.009 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.010 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.011 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.011 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.012 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:22.464 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:22.465 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:19:22.535 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.012 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:23.017 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:19:23.042 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.013 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.014 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.014 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.015 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.015 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.016 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.017 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:19:26.694 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.017 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:27.161 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:19:27.201 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.018 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.018 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.019 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.020 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.020 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.021 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.021 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.022 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:31.280 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:19:31.363 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.023 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:31.870 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.023 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.024 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.024 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.025 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.026 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.026 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.027 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:35.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:19:35.520 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.027 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:36.028 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.028 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.029 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.029 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.030 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.030 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.031 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.032 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:39.703 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.032 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:40.211 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.033 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.033 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.034 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.035 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.035 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.036 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.036 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:43.855 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.037 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:44.368 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.038 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.038 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.039 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.039 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.040 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.041 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.041 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:47.988 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.042 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:48.523 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.042 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.043 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.044 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.044 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.045 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.045 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.046 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:52.116 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.047 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:19:52.678 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.047 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.048 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.048 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.049 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.050 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.050 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.051 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:56.237 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.051 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:19:56.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:19:56.806 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:01.052 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.053 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.053 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.054 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.054 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.055 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.056 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.056 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:00.928 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:00.929 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:01.057 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:01.495 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:01.495 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:20:01.503 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.057 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.058 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.059 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.059 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.060 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.060 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.061 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:05.072 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:05.072 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:01.062 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:05.622 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.062 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:06.166 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.063 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.064 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.064 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.065 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.065 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.066 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.067 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.067 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:10.309 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.068 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:10.898 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:10.899 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:01.068 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:11.463 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:11.464 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:20:11.467 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.069 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.070 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.070 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.071 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.071 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.072 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.073 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:15.021 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:15.022 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:01.073 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:15.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:15.586 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:20:15.621 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.074 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.074 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.075 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.076 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.076 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.077 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.077 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:19.176 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:19.177 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:01.078 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:19.729 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:19.730 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:20:19.782 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.079 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.079 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.080 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.081 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.081 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.082 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.082 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.083 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:23.878 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:23.878 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:20:23.944 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.084 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:24.431 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:20:24.452 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.084 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.085 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.085 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.086 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.087 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.087 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.088 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:28.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:28.031 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:20:28.100 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.089 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:28.581 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:20:28.606 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.089 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.090 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.090 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.091 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.092 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.092 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.093 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:32.157 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:32.158 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:20:32.256 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.093 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:32.728 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:20:32.764 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.094 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.095 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.095 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.096 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.096 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.097 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.098 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.098 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:36.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:20:36.923 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.099 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:37.431 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.099 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.100 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.101 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.101 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.102 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.102 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.103 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:40.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:20:41.079 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.104 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:41.735 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.104 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.105 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.105 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.106 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.106 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.107 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.108 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:45.361 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.108 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:45.891 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.109 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.109 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.110 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.111 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.111 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.112 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.112 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:49.509 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.113 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:50.049 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.114 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.114 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.115 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.115 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.116 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.117 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.117 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:53.641 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.118 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:20:54.205 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.118 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.119 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.120 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.120 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.121 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.121 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.122 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:57.800 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.123 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:20:58.351 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:20:58.351 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:01.123 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.124 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.124 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.125 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.126 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.126 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.127 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:01.917 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.127 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:02.480 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:02.481 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:01.128 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.129 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.129 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.130 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.130 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.131 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.132 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.132 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:06.613 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:06.614 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:01.133 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:07.179 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:07.181 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:21:07.181 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.133 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.134 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.135 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.135 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.136 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.137 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.137 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:10.759 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:10.759 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:01.138 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:11.311 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:11.312 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:21:11.342 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.139 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.139 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.140 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.140 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.141 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.143 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:14.542 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.143 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:15.093 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:15.094 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:01.144 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.145 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.145 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.146 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.146 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.147 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.148 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:18.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.148 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:19.230 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:19.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:01.149 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.149 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.150 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.151 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.151 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.152 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.153 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.153 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:23.358 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:23.359 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:01.154 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:23.927 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:23.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:21:23.932 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.155 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.155 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.156 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.157 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.157 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.158 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.159 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:27.505 DEBUG:teuthology.orchestra.run:got remote process result: 22 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:27.507 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2022-02-17T21:01:01.159 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:28.060 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:28.061 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:21:28.092 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.160 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.160 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.161 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.162 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.162 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.163 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.163 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:31.658 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:31.660 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2022-02-17T21:01:01.164 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:32.210 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:32.211 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:21:32.253 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.164 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.165 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.166 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.166 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.167 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.167 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.168 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.169 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:36.340 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:21:36.341 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:21:36.409 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.169 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:36.891 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:21:36.918 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.170 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.171 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.171 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.172 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.172 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.173 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.174 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.174 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:41.126 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:21:41.178 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.175 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:41.683 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.175 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.176 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.176 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.177 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.178 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.178 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.179 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:45.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:21:45.333 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.180 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:45.841 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.180 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.181 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.182 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.182 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.183 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.183 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.184 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:49.389 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:21:49.488 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.185 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:49.995 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.185 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.186 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.187 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.187 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.188 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.189 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.189 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:53.643 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.190 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:54.150 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.190 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.191 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.192 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.192 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.193 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.193 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.194 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:21:57.784 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.195 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:21:58.305 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.195 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.196 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.196 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.197 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.198 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.198 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.199 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:01.913 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.199 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:02.464 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.200 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.201 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.201 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.202 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.202 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.203 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.204 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.204 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:06.633 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:22:06.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:01.205 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:07.198 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:07.205 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.205 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:07.726 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.206 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.207 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.207 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.208 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.208 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.209 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.210 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.210 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:11.789 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:22:11.881 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.211 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:12.387 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.211 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.212 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.212 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.213 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.214 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.214 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.215 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:16.024 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.215 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:16.543 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.216 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.216 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.217 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.217 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.218 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.219 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.219 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:20.165 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.220 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:20.698 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.220 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.221 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.222 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.222 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.223 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.223 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.224 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:24.294 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.224 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:24.855 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.225 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.226 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.226 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.227 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.227 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.228 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.228 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:28.438 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.229 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:29.014 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:22:29.015 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:01.230 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:29.582 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:22:29.583 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:22:29.594 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.230 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.231 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.231 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.232 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.232 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.233 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.234 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:33.161 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:22:33.162 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:01.234 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:33.710 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:22:33.711 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:22:33.749 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.235 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.235 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.236 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.237 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.237 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.238 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.238 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.239 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:37.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:22:37.852 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:22:37.904 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.239 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:38.405 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:22:38.411 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.240 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.240 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.241 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.242 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.242 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.243 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.243 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:41.986 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:22:41.987 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:22:42.062 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.244 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:42.554 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:22:42.568 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.244 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.245 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.245 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.246 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.247 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.247 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.248 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:46.128 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:22:46.129 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:22:46.218 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.248 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:46.680 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:22:46.725 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.249 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.249 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.250 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.251 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.251 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.252 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.252 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.253 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:51.014 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.253 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:51.536 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.254 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.254 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.255 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.256 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.256 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.257 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.257 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:55.138 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.258 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:22:55.692 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.258 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.259 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.260 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.260 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.261 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.261 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.262 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:59.293 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.263 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:22:59.844 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:22:59.845 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:01.263 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.264 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.264 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.265 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.266 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.266 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.267 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:03.432 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.267 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:03.982 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:03.983 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2022-02-17T21:01:01.268 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.269 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.269 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.270 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.270 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.271 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.272 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.272 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:08.115 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:08.116 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2022-02-17T21:01:01.273 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:08.666 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.273 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.274 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.275 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.275 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.276 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.276 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.277 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:12.298 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:12.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:01.277 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:12.865 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.278 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.279 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.279 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.280 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.280 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.281 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.281 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:16.421 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:16.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:01.282 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:16.990 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:16.991 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:23:17.019 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.283 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.283 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.284 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.284 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.285 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.286 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.286 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:20.587 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:20.588 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:01.287 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:21.137 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:21.139 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:23:21.176 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.287 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.288 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.289 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.289 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.290 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.290 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.291 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.291 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:25.274 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:25.275 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:23:25.335 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.292 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:25.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:23:25.843 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.293 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.293 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.294 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.294 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.295 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.295 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.296 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:29.408 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:29.409 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:23:29.490 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.297 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:29.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:23:29.997 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.297 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.298 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.298 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.299 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.300 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.300 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.301 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.301 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:34.096 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:23:34.153 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.302 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:34.660 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.302 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.303 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.304 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.304 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.305 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.305 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.306 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:38.397 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.307 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:38.923 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.307 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.308 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.309 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.309 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.310 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.310 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.311 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:42.530 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.311 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:43.080 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.312 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.313 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.313 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.314 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.314 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.315 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.316 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:46.659 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.316 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:47.225 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:47.225 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:01.317 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.317 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.318 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.319 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.319 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.320 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.320 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:50.801 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.321 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:51.351 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:51.352 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:01.321 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.322 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.323 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.323 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.324 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.331 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.331 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.332 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:55.487 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:55.489 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:01.333 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:23:56.040 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:56.041 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:23:56.054 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.333 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.334 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.334 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.335 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.335 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.336 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.337 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:23:59.625 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:23:59.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:01.337 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:00.199 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:24:00.200 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:24:00.209 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.338 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.338 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.339 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.340 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.340 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.341 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:03.326 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.341 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:03.892 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:24:03.893 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:01.342 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.342 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.343 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.344 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.344 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.345 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.345 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:07.472 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.346 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:08.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:24:08.024 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:01.347 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.347 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.348 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.348 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.349 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.350 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.350 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.351 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.352 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:12.653 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:24:12.654 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:24:12.737 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.352 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:13.219 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:24:13.245 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.353 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.353 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.354 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.355 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.355 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.356 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.357 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:16.804 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:24:16.804 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:24:16.894 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.357 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:17.356 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:24:17.402 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.358 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.358 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.359 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.360 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.360 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.361 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.361 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.362 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:21.507 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:24:21.561 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.363 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:22.070 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.363 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.364 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.364 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.365 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.365 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.366 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.367 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:25.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:24:25.720 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.367 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:26.227 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.368 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.368 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.369 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.369 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.370 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.371 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.371 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:29.778 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:24:29.878 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.372 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:30.385 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.372 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.373 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.374 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.374 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.375 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.375 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.376 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:34.079 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:24:34.149 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.376 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:34.655 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.377 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.377 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.378 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.379 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.379 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.380 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.381 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:38.220 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:24:38.300 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.381 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:38.806 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.382 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.382 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.383 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.384 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.384 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.385 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.385 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:24:42.457 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.386 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:42.965 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.387 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.387 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.388 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.388 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.389 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.390 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.390 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:46.589 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.391 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:47.122 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.391 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.392 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.392 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.393 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.394 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.394 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.395 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:50.718 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.395 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:24:51.282 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.396 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.397 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.397 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.398 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.398 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.399 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.399 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:54.872 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.400 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:55.423 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:24:55.424 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:01.401 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.401 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.402 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.402 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.403 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.404 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.404 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:59.003 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.405 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:24:59.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:24:59.554 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:01.405 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:00.104 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:00.105 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:25:00.106 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.406 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.407 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.407 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.408 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.409 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.409 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.410 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:03.765 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:03.766 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:01.411 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:04.317 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:04.318 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:25:04.319 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.411 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.412 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.413 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.413 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.414 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.415 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.415 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:07.886 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:07.887 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2022-02-17T21:01:01.416 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:08.452 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:08.453 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:25:08.475 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.417 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.417 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.418 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.418 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.419 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.420 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.420 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.421 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:12.574 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:12.575 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:25:12.630 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.421 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.422 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.422 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.423 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.424 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.424 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.425 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.425 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:16.713 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:16.714 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:25:16.790 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.426 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:17.267 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:25:17.298 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.427 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.427 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.428 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.428 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.429 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.430 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.430 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:25:20.947 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.431 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:21.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:25:21.455 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.431 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:21.960 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.432 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.433 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.433 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.434 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.434 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.435 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.435 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:25.531 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:25:25.608 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.436 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:26.116 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.437 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.437 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.438 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.438 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.439 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.440 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.440 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:29.684 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:25:29.763 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.441 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:30.271 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.441 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.442 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.442 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.443 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.444 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.444 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.445 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:34.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.445 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:34.557 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.446 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.447 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.447 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.448 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.448 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.449 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.449 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:38.191 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.450 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:38.717 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.451 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.451 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.452 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.452 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.453 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.453 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.454 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:42.313 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.455 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:42.874 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.455 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.456 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.456 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.457 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.458 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.458 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.459 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:46.444 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.459 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:46.997 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:46.998 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:01.460 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.460 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.461 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.462 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.462 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.463 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.463 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.464 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:51.139 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:51.140 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:01.465 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:51.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:51.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:25:51.692 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.465 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.466 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.466 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.467 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.467 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.468 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.469 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:55.269 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:55.270 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:01.469 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:55.837 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:55.839 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:25:55.851 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.470 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.470 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.471 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.472 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.472 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.473 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.473 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:25:59.413 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:59.414 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:01.474 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:25:59.966 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:25:59.967 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:26:00.009 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.475 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.475 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.476 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.476 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.477 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.477 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.478 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.479 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:04.180 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:04.181 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:26:04.232 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.479 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:04.733 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:26:04.739 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.480 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.480 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.481 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.481 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.482 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.483 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.483 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:08.298 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:08.299 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:26:08.385 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.484 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:08.864 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:26:08.891 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.484 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.485 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.485 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.486 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.487 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.487 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.488 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.488 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:26:12.987 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:12.988 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:01.489 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:13.552 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.489 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.490 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.491 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.491 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.492 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.492 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.493 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:26:17.126 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.494 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:26:17.675 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:17.676 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:01.494 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.495 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.495 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.496 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.497 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.497 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.498 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:26:21.267 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.498 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:26:21.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:21.820 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2022-02-17T21:01:01.499 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:22.371 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:22.371 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:26:22.372 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.500 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.500 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.501 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.501 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.502 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.503 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.503 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:26:25.954 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:25.955 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2022-02-17T21:01:01.504 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:26.507 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:26.508 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:26:26.529 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.504 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.505 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.506 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.506 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.507 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.508 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.508 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.509 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:30.641 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:30.641 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:26:30.690 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.509 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:31.193 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:26:31.197 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.510 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.511 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.511 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.512 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.512 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.513 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.513 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:34.956 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:34.956 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:26:35.015 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.514 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.515 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.515 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.516 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.516 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.517 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.518 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.518 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:39.084 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:26:39.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:26:39.170 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.519 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:39.650 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:26:39.678 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.519 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.520 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.521 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.521 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.522 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.522 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.523 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:26:43.328 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.524 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:43.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:26:43.834 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.524 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:44.341 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.525 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.525 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.526 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.527 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.527 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.528 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.528 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:47.920 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:26:47.988 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.529 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:48.494 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.529 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.530 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.531 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.531 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.532 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.532 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.533 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:52.059 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:26:52.147 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.534 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:52.654 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.534 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.535 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.535 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.536 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.537 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.537 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.538 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:26:56.293 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.539 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:26:56.808 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.539 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.540 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.540 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.541 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.542 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.542 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.543 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:27:00.505 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.543 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:27:01.056 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:01.057 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:01.544 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.545 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.545 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.546 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.547 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.547 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.548 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:27:04.624 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.548 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:27:05.187 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:05.188 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:01.549 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.550 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.550 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.551 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.551 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.552 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.553 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.553 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:27:09.310 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:09.311 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:01.554 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:09.874 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:09.875 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:27:09.884 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.554 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.555 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.555 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.556 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.557 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.557 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.558 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:27:13.448 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:13.449 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:01.558 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:14.000 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:14.000 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:27:14.038 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.559 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.560 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.560 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.561 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.561 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.562 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.563 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:27:17.592 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:17.593 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:01.563 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:18.162 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:18.163 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:27:18.197 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.564 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.564 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.565 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.566 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.566 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.567 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.568 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.568 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:22.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:22.291 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:27:22.352 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.569 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:22.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:27:22.858 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.569 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.570 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.571 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.571 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.572 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.572 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:27:26.022 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:26.024 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:01.573 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:26.588 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:26.589 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:27:26.600 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.574 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.574 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.575 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.575 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.576 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.577 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.577 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:27:30.160 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:30.161 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:01.578 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:30.714 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:30.715 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:27:30.755 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.578 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.579 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.580 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.580 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.581 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.581 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.582 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.583 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:34.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:34.858 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:27:34.910 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.583 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:35.408 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:27:35.418 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.584 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.584 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.585 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.586 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.586 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.587 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.587 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:38.981 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:38.982 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:27:39.064 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.588 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:39.555 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:27:39.571 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.589 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.589 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.590 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.590 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.591 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.592 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.592 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:43.127 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:43.128 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:27:43.221 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.593 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:43.680 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:27:43.730 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.593 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.594 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.595 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.595 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.596 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.596 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.597 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:47.373 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:27:47.374 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:27:47.463 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.598 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:47.940 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:27:47.970 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.598 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.599 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.600 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.600 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.601 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.601 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.602 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.603 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:52.061 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:27:52.125 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.603 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.604 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.605 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.605 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.606 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.606 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.607 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.608 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:56.221 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:27:56.284 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.608 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:27:56.793 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.609 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.609 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.610 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.611 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.611 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.612 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.612 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:00.371 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:28:00.445 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.613 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:00.953 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.614 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.614 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.615 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.615 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.616 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.616 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.617 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:28:04.605 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.618 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:05.112 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.618 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.619 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.619 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.620 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.620 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.621 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.622 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:28:08.892 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.622 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:09.398 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.623 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.624 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.625 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.626 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.626 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.627 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.628 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:28:13.018 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.628 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:13.459 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:28:13.460 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:28:13.555 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.629 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:14.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:28:14.062 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.629 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.630 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.631 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.631 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.632 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.632 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.633 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.633 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:18.144 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:28:18.146 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:28:18.229 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.634 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:18.714 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:28:18.737 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.635 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.635 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.636 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.636 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.637 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.638 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.638 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:22.287 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:28:22.287 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:28:22.384 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.639 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:22.837 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:28:22.892 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.639 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.640 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.641 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.641 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.642 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.642 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.643 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:28:26.544 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.643 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:26.994 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:28:27.052 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.644 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:27.560 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.645 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.645 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.646 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.646 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.647 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.648 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.648 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:31.146 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:28:31.207 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.649 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:31.714 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.649 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.650 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.650 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.651 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.652 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.652 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.653 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:35.283 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:28:35.365 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.653 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:35.873 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.654 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.655 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.655 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.656 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.656 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.657 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.657 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:28:39.527 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.658 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:40.036 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.659 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.659 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.660 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.660 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.661 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.662 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.662 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:28:43.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.663 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:44.192 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.664 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.664 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.665 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.665 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.666 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.666 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.667 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:28:47.810 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.668 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:48.347 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.668 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.669 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.669 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.670 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.671 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.671 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.672 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:28:51.953 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.672 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:28:52.504 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.673 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.673 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.674 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.675 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.675 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.676 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.676 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:28:56.089 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.677 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:28:56.642 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:28:56.643 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:01.678 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.678 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.679 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.679 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.680 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.681 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.681 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:29:00.208 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.682 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:29:00.773 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:00.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:01.682 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.683 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.684 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.684 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.685 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.685 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.686 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.686 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.687 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:05.517 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:05.518 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:29:05.584 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.688 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:06.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:29:06.090 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.688 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.689 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.689 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.690 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.691 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.691 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.692 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:09.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:09.666 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:29:09.742 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.693 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:10.234 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:29:10.247 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.693 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.694 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.694 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.695 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.696 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.696 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.697 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:13.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:13.820 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:29:13.892 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.697 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:14.370 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:29:14.399 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.698 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.699 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.699 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.700 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.700 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.701 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.702 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:29:18.055 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.702 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:18.520 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:29:18.562 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.703 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.704 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.704 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.705 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.705 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.706 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.707 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.707 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:22.640 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:29:22.721 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.708 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:23.228 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.708 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.709 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.710 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.710 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.711 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.711 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.712 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:26.794 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:29:26.876 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.713 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:27.384 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.713 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.714 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.714 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.715 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.716 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.716 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.717 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:29:31.143 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.718 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:29:31.710 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:31.711 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:01.718 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.719 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.720 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.720 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.721 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.721 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.722 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:29:35.283 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.723 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:29:35.833 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:35.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:01.723 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.724 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.724 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.725 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.726 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.726 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.727 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.727 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:29:39.975 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:39.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:01.728 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:40.526 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:40.526 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:29:40.547 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.729 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.729 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.730 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.730 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.731 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.732 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.732 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:29:44.117 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:44.118 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:01.733 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:44.669 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:44.670 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:29:44.704 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.733 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.734 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.735 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.735 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.736 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.736 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.737 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.738 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:48.814 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:48.815 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:29:48.862 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.738 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:49.363 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:29:49.368 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.739 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.739 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.740 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.741 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.741 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.742 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.742 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:52.936 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:29:52.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:29:53.023 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.743 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:53.503 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:29:53.530 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.744 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.744 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.745 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.745 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.746 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.747 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.747 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:29:57.184 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.748 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:57.628 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:29:57.690 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.748 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:29:58.196 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.749 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.750 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.750 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.751 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.751 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.752 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:01.278 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:01.279 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:30:01.372 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.752 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:01.841 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:30:01.878 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.753 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.754 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.754 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.755 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.755 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.756 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.757 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.757 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:05.962 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:30:06.032 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.758 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:06.539 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.758 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.759 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.760 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.760 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.761 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.761 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.762 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:10.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:30:10.190 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.762 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:10.696 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.763 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.764 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.764 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.765 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.765 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.766 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.767 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:30:14.339 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.767 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:14.852 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.768 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:15.349 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:30:15.358 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.768 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.769 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.770 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.770 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.771 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.771 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.772 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:30:18.914 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:18.915 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:01.773 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:19.478 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:19.479 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:30:19.513 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.773 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.774 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.774 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.775 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.776 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.776 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.777 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.777 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:23.604 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:23.605 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:30:23.667 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.778 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.779 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.779 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.780 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.780 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.781 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.782 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:30:27.373 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:27.373 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:01.782 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:27.937 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:27.938 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:30:27.956 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.783 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.783 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.784 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.785 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.785 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.786 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.786 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:30:31.512 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:31.513 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:01.787 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:32.065 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:32.066 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:30:32.113 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.788 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.788 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.789 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.789 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.790 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.791 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.791 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.792 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:36.214 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:36.215 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:30:36.271 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.792 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:36.765 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:30:36.777 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.793 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.794 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.794 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.795 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.795 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.796 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.797 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:40.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:40.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:30:40.427 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.797 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:40.914 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:30:40.933 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.798 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.798 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.799 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.800 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.800 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.801 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.801 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:44.486 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:44.487 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:30:44.580 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.802 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:45.039 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:30:45.088 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.803 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.803 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.804 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.805 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.805 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.806 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.806 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.807 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:49.182 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:30:49.244 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.808 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:49.750 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.808 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.809 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.809 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.810 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.810 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.811 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.812 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:53.309 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:30:53.399 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.812 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:53.905 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.813 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.813 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.814 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.815 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.815 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.816 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:57.004 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:30:57.007 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:30:57.074 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.816 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:30:57.579 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:30:57.581 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.817 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.818 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.818 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.819 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.819 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.820 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.821 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:01.147 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:01.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:31:01.229 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.821 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:01.702 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:31:01.735 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.822 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.822 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.823 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.824 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.824 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.825 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.825 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:05.283 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:05.284 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:31:05.384 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.826 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:05.851 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:31:05.891 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.827 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.827 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.828 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.828 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.829 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.830 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.830 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.831 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:09.972 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:31:10.045 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.831 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:10.553 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.832 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.833 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.833 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.834 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.834 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.835 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.836 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:14.124 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:31:14.205 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.836 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:14.713 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.837 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.837 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.838 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.839 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.839 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.840 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.840 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:31:18.364 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.841 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:18.870 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.842 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.842 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.843 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.843 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.844 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.844 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.845 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:22.502 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.846 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:23.027 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.846 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.847 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.848 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.848 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.849 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.849 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.850 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:26.780 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.851 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:27.335 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:27.336 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2022-02-17T21:01:01.851 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.852 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.852 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.853 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.854 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.854 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.855 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:30.916 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.855 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:31.479 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:31.480 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:01.856 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.856 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.857 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.858 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.858 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.859 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.860 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:35.051 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.860 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:35.599 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:35.600 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:01.861 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.861 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.862 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.863 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.863 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.864 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.864 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.865 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:39.746 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:39.747 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:01.866 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:40.295 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:40.296 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:31:40.314 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.866 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.867 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.867 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.868 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.868 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.869 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.870 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:43.872 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:43.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:01.870 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:44.446 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:44.447 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:31:44.468 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.871 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.871 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.872 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.873 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.873 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.874 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.874 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:31:48.027 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:48.028 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:01.875 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:48.579 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:31:48.580 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:31:48.623 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.876 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.876 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.877 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.877 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.878 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.879 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.879 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.880 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.880 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:53.352 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:31:53.425 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.881 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:53.932 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.881 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.882 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.883 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.883 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.884 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.885 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.885 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:57.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:31:57.588 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.886 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:31:58.095 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.886 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.887 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.888 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.888 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.889 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.889 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.890 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:01.734 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.890 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:02.255 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.891 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.892 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.892 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.893 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.893 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.894 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.895 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:05.876 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.895 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:06.408 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.896 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.896 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.897 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.898 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.898 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.899 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.899 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:10.008 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.900 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:10.570 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.901 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.901 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.902 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.902 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.903 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.904 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.904 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:14.328 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.905 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:14.853 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.905 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.906 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.907 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.907 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.908 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.908 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.909 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.910 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.910 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:19.457 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:19.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:32:19.517 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.911 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.911 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.912 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.913 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.913 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.914 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.914 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.915 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:23.603 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:23.604 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:32:23.673 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.916 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:24.160 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:32:24.179 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.916 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.917 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.917 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.918 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.918 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.919 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.920 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:27.746 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:27.747 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:32:27.830 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.920 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:28.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:32:28.342 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.921 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.921 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.922 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.923 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.923 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.924 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.924 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:31.900 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:31.901 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:32:31.996 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.925 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:32.453 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:32:32.502 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.926 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.926 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.927 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.927 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.928 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.928 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.929 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:32:36.155 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.930 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:36.603 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:32:36.662 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.930 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:37.170 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.931 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.931 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.932 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.932 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.933 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.934 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.934 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:40.977 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.935 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:41.505 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.935 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.936 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.937 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.937 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.938 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.939 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.939 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:45.096 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.940 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:45.655 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:45.656 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2022-02-17T21:01:01.940 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.941 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.941 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.942 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.943 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.943 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.944 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.944 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:49.775 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:49.775 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:01.945 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.946 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.946 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.947 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.947 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.948 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.949 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.949 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:53.896 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:53.897 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:01.950 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:54.471 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:54.472 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:32:54.485 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.950 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.951 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.952 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.952 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.953 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.953 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.954 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:32:58.049 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:58.050 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:01.955 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:58.785 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:32:58.786 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:32:58.865 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.955 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:32:59.351 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:32:59.372 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.956 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.956 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.957 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:01.958 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.958 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:01.959 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.959 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:02.924 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:33:02.925 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:33:03.023 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:01.960 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:03.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:33:03.530 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.961 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:01.961 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.962 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.962 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.963 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.964 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.964 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.965 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:07.625 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:33:07.690 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.965 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:08.196 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.966 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.967 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.967 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.968 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.968 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.969 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.970 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:11.767 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:33:11.844 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.970 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:12.352 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.971 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.971 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.972 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.972 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.973 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.974 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.974 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:15.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:33:16.004 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.975 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:16.511 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.975 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.976 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.977 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:01.977 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.978 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:01.978 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.979 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:20.253 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:33:20.344 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:01.980 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:20.850 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.980 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:01.981 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.981 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.982 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.982 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.983 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.984 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:33:24.488 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.984 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:25.005 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.985 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.985 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.986 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.987 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.987 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.988 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:01.988 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:33:28.627 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.989 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:29.160 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.990 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.990 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:01.991 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.991 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:01.992 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.993 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.993 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:33:32.751 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.994 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:33.316 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:01.994 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:01.995 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:01.996 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:01.996 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:01.997 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:01.997 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:01.998 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:33:36.888 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:01.999 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:33:37.443 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:33:37.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:01.999 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.000 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.000 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.001 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.001 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.002 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.003 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.003 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:33:41.759 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:33:41.760 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:02.004 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:42.313 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:33:42.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:33:42.353 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.004 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.005 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.006 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.006 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.007 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.007 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.008 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.009 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:46.448 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:33:46.449 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:33:46.508 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.009 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:47.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:33:47.017 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.010 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.010 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.011 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.012 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.012 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.013 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.013 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:50.575 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:33:50.575 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:33:50.665 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.014 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:51.141 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:33:51.173 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.015 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.015 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.016 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.016 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.017 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.017 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.018 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.019 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:55.261 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:33:55.336 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.019 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:55.845 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.020 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.020 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.021 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.022 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.022 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.023 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.023 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:33:59.395 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:33:59.494 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.024 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:00.002 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.025 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.025 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.026 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.026 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.027 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.028 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.028 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:34:03.649 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.029 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:04.155 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.029 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.030 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.031 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.031 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.032 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.032 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.033 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:08.038 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.034 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:08.587 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.034 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.035 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.035 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.036 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.037 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.037 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.038 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:12.165 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.038 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:12.729 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:34:12.731 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:02.039 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.040 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.040 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.041 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.041 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.042 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.043 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:16.296 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.043 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:16.846 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:34:16.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:02.044 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.045 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.045 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.046 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.046 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.047 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.048 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.048 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.049 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:21.467 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:34:21.468 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:34:21.562 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.049 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:22.021 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:34:22.069 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.050 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:22.576 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.051 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.051 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.052 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.052 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.053 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.054 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:25.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:34:25.820 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:34:25.901 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.054 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:26.372 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:34:26.408 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.055 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.056 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.056 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.057 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.057 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.058 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.059 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:29.967 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:34:29.968 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:34:30.059 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.059 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:30.520 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:34:30.566 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.060 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:31.073 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.060 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.061 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.062 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.062 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.063 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.063 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.064 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:34.655 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:34:34.723 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.065 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:35.231 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.065 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.066 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.067 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.067 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.068 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.069 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.069 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:38.783 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:34:38.883 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.070 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:39.390 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.070 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.071 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.072 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.072 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.073 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.074 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.074 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:43.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.075 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:43.545 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.075 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.076 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.077 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.077 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.078 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.078 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.079 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:47.366 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.080 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:47.930 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:34:47.931 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:02.080 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.081 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.081 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.082 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.083 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.083 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.084 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:51.517 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.085 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:52.068 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:34:52.068 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:02.085 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.086 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.087 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.087 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.088 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.088 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.089 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.090 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:34:56.213 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:34:56.214 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:02.090 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:34:56.761 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:34:56.762 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:34:56.782 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.091 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.091 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.092 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.093 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.093 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.094 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.095 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.095 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:00.897 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:00.898 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:35:00.937 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.096 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.096 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.097 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.098 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.098 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.099 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.099 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.100 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:05.030 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:05.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:35:05.099 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.101 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:05.596 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:35:05.606 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.101 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.103 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.104 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.104 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.105 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.105 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.106 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:09.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:09.170 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:35:09.259 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.107 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:09.722 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:35:09.766 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.107 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.108 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.108 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.109 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.110 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.110 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.111 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.112 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:14.135 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:35:14.214 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.112 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:14.721 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.113 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.113 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.114 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.115 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.115 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.116 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.116 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:18.287 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:35:18.373 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.117 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:18.880 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.118 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.118 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.119 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.119 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.120 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.121 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.121 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:22.534 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.122 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:23.039 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.122 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.129 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.130 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.131 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.131 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.132 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.132 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:26.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.133 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:27.201 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.134 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.134 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.135 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.135 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.136 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.137 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.137 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:30.794 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.138 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:31.347 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:31.348 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:02.139 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.139 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.140 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.140 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.141 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.142 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.142 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:34.914 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.143 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:35.480 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:35.481 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:02.143 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.144 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.145 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.145 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.146 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.147 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.147 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.148 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:39.588 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:39.589 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:02.148 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:40.153 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:40.154 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:35:40.173 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.149 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.150 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.150 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.151 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.151 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.152 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.153 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:43.862 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:43.863 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2022-02-17T21:01:02.153 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:44.414 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:44.415 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:35:44.427 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.154 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.155 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.155 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.156 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.156 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.157 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.158 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:48.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:48.049 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2022-02-17T21:01:02.158 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.159 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.159 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.160 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.161 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.161 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.162 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.162 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:35:52.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:52.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:02.163 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:52.739 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:52.740 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:35:52.746 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.164 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.164 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.165 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.166 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.166 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.167 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.167 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:35:56.300 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:56.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:02.168 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:35:56.865 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:35:56.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:35:56.906 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.169 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.169 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.170 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.170 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.171 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.172 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.172 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.173 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:00.996 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:36:00.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:36:01.066 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.173 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:01.561 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:36:01.573 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.174 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.175 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.175 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.176 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.177 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.177 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.178 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:05.126 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:36:05.127 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:36:05.224 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.178 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:05.677 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:36:05.732 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.179 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.180 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.180 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.181 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.181 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.182 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.183 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.183 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:09.817 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:36:09.892 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.184 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:10.399 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.185 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.185 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.186 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.187 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.187 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.188 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.188 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:36:14.336 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.189 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:14.841 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.190 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.190 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.191 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.192 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.192 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.193 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.193 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:36:18.468 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.194 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:18.997 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.195 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.195 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.196 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.197 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.197 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.198 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.199 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.199 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:23.055 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:36:23.056 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:36:23.155 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.200 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:23.635 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:36:23.636 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:36:23.669 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.200 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.201 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.202 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.202 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.203 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.203 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.204 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.205 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:27.762 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:36:27.763 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:36:27.825 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.205 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.206 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.207 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.207 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.208 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.208 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.209 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.210 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:31.901 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:36:31.902 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:36:31.979 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.210 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:32.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:36:32.486 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.211 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.212 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.212 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.213 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.213 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.214 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.215 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:36.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:36:36.037 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:36:36.135 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.215 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:36.601 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:36:36.641 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.216 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.217 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.217 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.218 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.218 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.219 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.220 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.220 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:40.729 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:36:40.799 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.221 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:41.305 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.221 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.222 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.223 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.223 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.224 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.224 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.225 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:45.038 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:36:45.120 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.226 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:45.626 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.226 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.227 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.227 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.228 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.229 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.229 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.230 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:36:49.275 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.230 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:49.783 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.231 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.232 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.232 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.233 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.233 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.234 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.234 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:36:53.419 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.235 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:53.938 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.236 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.236 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.237 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.237 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.238 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.239 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.239 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:36:57.546 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.240 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:36:58.092 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.240 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.241 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.242 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.242 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.243 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.243 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.244 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:01.700 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.244 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:02.250 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:02.250 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:02.245 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.246 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.246 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.247 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.247 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.248 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.249 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:05.815 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.249 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:06.383 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:06.383 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:02.250 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.250 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.251 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.251 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.252 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.253 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.253 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.254 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:10.620 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:10.621 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:02.254 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:11.187 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:11.188 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:37:11.218 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.255 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.256 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.256 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.257 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.257 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.258 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.258 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.259 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:15.314 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:15.315 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:37:15.378 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.260 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:15.883 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:37:15.886 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.260 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.261 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.261 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.262 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.263 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.263 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.264 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:19.450 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:19.451 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:37:19.541 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.264 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:20.003 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:37:20.048 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.265 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.266 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.266 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.267 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.267 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.268 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.269 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.269 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:24.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:37:24.204 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.270 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:24.712 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.270 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.271 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.271 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.272 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.273 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.273 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.274 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:28.267 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:37:28.360 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.274 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:28.868 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.275 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.276 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.276 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.277 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.277 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.278 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.279 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:32.422 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:37:32.521 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.279 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:33.027 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.280 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.280 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.281 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.281 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.282 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.283 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.283 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:36.760 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.284 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:37.294 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.284 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.285 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.286 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.286 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.287 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.288 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.288 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:40.875 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.289 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:41.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:41.442 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:02.289 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.290 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.291 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.291 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.292 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.292 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.293 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:45.015 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.293 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:45.566 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:45.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2022-02-17T21:01:02.294 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.295 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.295 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.296 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.297 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.297 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.298 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.298 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:49.702 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:49.703 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:02.299 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:50.255 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:50.256 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:37:50.269 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.300 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.300 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.301 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.301 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.302 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.303 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.303 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:53.836 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:53.837 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:02.304 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:54.411 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:54.412 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:37:54.426 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.304 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.305 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.306 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.306 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.307 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.307 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.308 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:37:57.986 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:57.987 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:02.309 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:37:58.538 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:37:58.539 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:37:58.583 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.309 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.310 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.310 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.311 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.312 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.312 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.313 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.313 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:02.698 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:02.699 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:38:02.740 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.314 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.315 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.315 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.316 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.316 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.317 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.318 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.318 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:06.919 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:06.920 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:38:07.008 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.319 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:07.486 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:38:07.515 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.320 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.320 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.321 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.321 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.322 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.322 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.323 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:38:11.170 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.324 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:11.617 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:38:11.676 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.324 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.325 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.325 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.326 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.327 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.327 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.328 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.328 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:15.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:38:15.834 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.329 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:16.341 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.329 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.330 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.331 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.331 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.332 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.332 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.333 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:38:19.993 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.334 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:20.507 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.334 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.335 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.335 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.336 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.337 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.337 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.338 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.338 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:24.571 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:24.572 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:38:24.664 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.339 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:25.134 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:38:25.171 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.340 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.340 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.341 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.341 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.342 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.343 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.343 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:38:28.777 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:28.778 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:02.344 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:29.327 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:29.327 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:38:29.336 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.344 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.345 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.346 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.346 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.347 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.348 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:38:32.501 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.348 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:38:33.056 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:33.057 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:02.349 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.349 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.350 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.351 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.351 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.352 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.352 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.353 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:38:37.204 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:37.205 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:02.354 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:37.757 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:37.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:38:37.760 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.354 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.355 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.356 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.356 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.357 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.357 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.358 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:38:41.319 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:41.320 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:02.359 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:41.882 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:41.883 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:38:41.916 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.359 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.360 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.360 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.361 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.361 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.362 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.363 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.363 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:46.014 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:46.015 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:38:46.075 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.364 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:46.580 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:38:46.581 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.364 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.365 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.366 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.366 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.367 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.368 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.368 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:50.173 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:50.174 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:38:50.238 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.369 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:50.726 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:38:50.745 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.369 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.370 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.370 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.371 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.372 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.372 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:38:53.935 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:53.936 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:02.373 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:54.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:54.491 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:38:54.531 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.373 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.374 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.375 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.375 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.376 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.376 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.377 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.377 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:58.624 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:38:58.625 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:38:58.693 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.378 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:38:59.176 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:38:59.200 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.379 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.379 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.380 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.380 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.381 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.382 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.382 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:02.764 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:02.765 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:39:02.853 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.383 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:03.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:39:03.359 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.383 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.384 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.385 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.385 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.386 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.386 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.387 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:06.911 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:06.912 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:39:07.012 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.387 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:07.464 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:39:07.519 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.388 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.389 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.389 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.390 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.390 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.391 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.391 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:39:11.167 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.392 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:11.614 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:39:11.675 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.393 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:12.181 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.393 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.394 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.394 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.395 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.396 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.396 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.397 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:15.747 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:39:15.827 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.397 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:16.335 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.398 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.399 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.399 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.400 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.400 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.401 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.401 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:19.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.402 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:20.493 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.403 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.403 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.404 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.404 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.405 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.406 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.406 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:24.228 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.407 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:24.763 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.407 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.408 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.408 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.409 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.410 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.410 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.411 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:28.362 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.412 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:28.921 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.412 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.413 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.413 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.414 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.415 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.415 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.416 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:32.499 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.416 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:33.052 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:33.053 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:02.417 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.418 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.418 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.419 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.420 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.420 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.421 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:36.632 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.421 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:37.199 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:37.201 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:02.422 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.423 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.423 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.424 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.425 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.425 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.426 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.426 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:41.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:41.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:02.427 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:41.897 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:41.898 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:39:41.898 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.428 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.428 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.429 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.430 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.430 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.431 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.431 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:45.485 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:45.486 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:02.432 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:46.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:46.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:39:46.056 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.433 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.433 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.434 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.434 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.435 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.436 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:49.217 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.436 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:49.768 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:49.769 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:02.437 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.437 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.438 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.439 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.439 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.440 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.440 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.441 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:53.927 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:53.929 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:02.442 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:54.479 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.442 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.443 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.443 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.444 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.445 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.445 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.446 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:39:58.060 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:58.061 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:02.446 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:39:58.630 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:39:58.631 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:39:58.633 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.447 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.448 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.448 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.449 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.449 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.450 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.450 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:40:02.195 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:40:02.196 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:02.451 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:02.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:40:02.749 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:40:02.792 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.452 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.452 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.453 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.453 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.454 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.455 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.455 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.456 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:06.888 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:40:06.889 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:40:06.950 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.456 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:07.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:40:07.458 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.457 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.458 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.458 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.459 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.459 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.460 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.460 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:11.007 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:40:11.008 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:40:11.107 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.461 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:11.574 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:40:11.615 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.462 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.462 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.463 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.463 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.464 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.464 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.465 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.466 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:15.792 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:40:15.883 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.466 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:16.391 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.467 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.468 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.468 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.469 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.469 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.470 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.470 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:40:20.034 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.471 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:20.546 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.472 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.472 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.473 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.473 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.474 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.475 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.475 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:40:24.177 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.476 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:24.705 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.476 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.477 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.477 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.478 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.479 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.479 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.480 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.480 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.481 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:29.301 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:40:29.302 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:40:29.372 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.482 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:29.853 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:40:29.881 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.482 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.483 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.483 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.484 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.485 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.485 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.486 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:33.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:40:33.434 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:40:33.529 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.486 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:34.001 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:40:34.035 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.487 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.487 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.488 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.489 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.489 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.490 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.490 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.491 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:38.132 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:40:38.193 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.491 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.492 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.493 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.493 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.494 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.494 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.495 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:41.892 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:40:41.893 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:40:41.956 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.496 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:42.460 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:40:42.463 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.496 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.497 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.497 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.498 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.498 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.499 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.500 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:46.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:40:46.038 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:40:46.115 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.500 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:46.591 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:40:46.623 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.501 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.501 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.502 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.502 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.503 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.504 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.504 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:50.183 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:40:50.184 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:40:50.279 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.505 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:50.737 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:40:50.788 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.505 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:51.294 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.506 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.507 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.507 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.508 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.508 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.509 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.509 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:54.873 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:40:54.945 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.510 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:55.452 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.511 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.511 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.512 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.512 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.513 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.514 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.514 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:40:59.106 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.515 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:40:59.613 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.515 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.516 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.516 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.517 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.518 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.518 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.519 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:03.255 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.519 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:03.768 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.520 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.521 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.521 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.522 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.522 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.523 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.523 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:07.442 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.524 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:08.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:08.007 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:02.525 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.525 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.526 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.527 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.527 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.528 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.528 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:11.575 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.529 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:12.125 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:12.126 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:02.530 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.530 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.531 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.531 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.532 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.533 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.533 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.534 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:16.266 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:16.267 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:02.534 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:16.817 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:16.818 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:41:16.836 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.535 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.535 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.536 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.537 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.537 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.538 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.538 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:41:20.392 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:20.393 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:02.539 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:20.951 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:20.952 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:41:21.000 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.540 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:21.503 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:41:21.508 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.540 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.541 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.541 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.542 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.543 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.543 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.544 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:25.077 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:25.078 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:41:25.160 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.544 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:25.743 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:41:25.797 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.545 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:26.305 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.545 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.546 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.547 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.547 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.548 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.548 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.549 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:29.870 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:41:29.953 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.550 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:30.459 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.550 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.551 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.551 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.552 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.552 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.553 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.554 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:34.013 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:41:34.112 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.554 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:34.619 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.555 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.555 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.556 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.557 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.557 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.558 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.558 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:38.248 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.559 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:38.773 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.559 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.560 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.561 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.561 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.562 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.562 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.563 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:42.400 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.563 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:42.931 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.564 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.565 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.565 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.566 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.566 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.567 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.568 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:46.524 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.568 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:47.094 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.569 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.569 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.570 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.570 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.571 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.572 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.572 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:50.669 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.573 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:51.382 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:51.383 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:02.573 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:51.949 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:51.950 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:41:51.960 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.574 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.574 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.575 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.576 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.576 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.577 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.577 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:41:55.513 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:55.514 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:02.578 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:41:56.063 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:41:56.064 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:41:56.118 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.579 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.579 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.580 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.580 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.581 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.582 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.582 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.583 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:00.208 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:00.209 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:42:00.273 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.583 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:00.765 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:42:00.779 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.584 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.584 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.585 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.586 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.587 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.588 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.588 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:04.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:04.346 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:42:04.426 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.589 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:04.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:42:04.932 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.590 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.590 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.591 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.591 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.592 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.593 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.593 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:42:08.584 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.594 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:09.032 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:42:09.090 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.594 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:09.598 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.595 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.595 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.596 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.597 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.597 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.598 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.598 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:13.173 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:42:13.250 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.599 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:13.756 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.600 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.600 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.601 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.601 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.602 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.603 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:16.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:16.896 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:42:16.967 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.603 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:17.460 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:42:17.472 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.604 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.604 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.605 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.606 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.606 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.607 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.608 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:21.032 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:21.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:42:21.130 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.608 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:21.601 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:42:21.637 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.609 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.609 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.610 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.611 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.611 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.612 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.612 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:25.191 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:25.192 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:42:25.290 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.613 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:25.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:42:25.798 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.613 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.614 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.615 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.615 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.616 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.616 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.617 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.618 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:42:29.884 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.618 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:42:30.434 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:30.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:02.619 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.619 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.620 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.621 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.621 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.622 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.622 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:42:34.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.623 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:42:34.572 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:34.573 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:02.624 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:35.124 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:35.125 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:42:35.129 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.624 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.625 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.625 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.626 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.626 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.627 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.628 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:42:38.702 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:38.702 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:02.628 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:39.250 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:39.251 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:42:39.290 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.629 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.629 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.630 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.630 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.631 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.632 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.632 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.633 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:43.531 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:43.531 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:42:43.591 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.633 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:44.083 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:42:44.099 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.634 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.635 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.635 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.636 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.636 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.637 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.638 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:47.653 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:47.654 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:42:47.749 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.638 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:48.230 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:42:48.256 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.639 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.639 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.640 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.640 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.641 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.642 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.642 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:51.806 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:42:51.807 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:42:51.906 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.643 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:52.360 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:42:52.414 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.643 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.644 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.645 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.645 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.646 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.646 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.647 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.647 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:56.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:42:56.570 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.648 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:42:57.080 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.649 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.649 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.650 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.650 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.651 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.652 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.652 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:00.654 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:43:00.731 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.653 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:01.238 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.653 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.654 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.654 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.655 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.656 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.656 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.657 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:04.800 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:43:04.890 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.657 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:05.398 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.658 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.659 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.659 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.660 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.660 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.661 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.661 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:09.210 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.662 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:09.784 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:09.785 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:02.663 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.663 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.664 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.664 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.665 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.665 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.666 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:13.352 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.667 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:13.903 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:13.904 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:02.667 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.668 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.668 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.669 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.670 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.670 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.671 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.672 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:18.048 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:18.048 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:02.672 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:18.601 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:18.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:43:18.615 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.673 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.673 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.674 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.675 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.675 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.676 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.677 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:22.183 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:22.183 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:02.677 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:22.747 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:22.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:43:22.769 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.678 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.678 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.679 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.680 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.680 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.681 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.681 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:26.338 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:26.339 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:02.682 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:26.887 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:26.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:43:26.926 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.683 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.683 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.684 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.685 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.685 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.686 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.686 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.687 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:31.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:31.038 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:43:31.080 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.688 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.688 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.689 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.689 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.690 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.691 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.691 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.692 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.692 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:35.938 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:43:36.011 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.693 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:36.520 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.694 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.694 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.695 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.696 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.696 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.697 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.697 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:40.081 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:43:40.170 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.698 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:40.677 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.699 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.699 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.700 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.700 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.701 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.701 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.702 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:44.329 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.703 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:44.836 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.703 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.704 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.704 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.705 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.706 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.706 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.707 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:48.466 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.707 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:48.992 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.708 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.708 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.709 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.710 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.710 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.711 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.711 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:52.607 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.712 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:43:53.150 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.713 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.713 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.714 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.714 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.715 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.715 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.716 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:56.755 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.717 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:43:57.306 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:43:57.307 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:02.717 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.718 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.718 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.719 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.720 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.720 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.721 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.721 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:01.547 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:01.548 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:02.722 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:02.099 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:02.100 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:44:02.105 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.722 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.723 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.724 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.724 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.725 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.725 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.726 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:05.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:05.691 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2022-02-17T21:01:02.727 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.727 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.728 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.728 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.729 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.729 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.730 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.731 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:09.836 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:09.837 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:02.731 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:10.393 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:10.394 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:44:10.420 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.732 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.732 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.733 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.734 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.734 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.735 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.736 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.736 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:14.532 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:14.533 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:44:14.580 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.737 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:15.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:44:15.088 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.737 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.738 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.739 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.739 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.740 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.740 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.741 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:18.663 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:18.664 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:44:18.739 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.741 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:19.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:44:19.246 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.742 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.743 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.743 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.744 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.744 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.745 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.746 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:22.927 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:22.928 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:44:23.014 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.746 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:23.478 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:44:23.524 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.747 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.747 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.748 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.748 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.749 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.750 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.750 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.751 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:27.621 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:44:27.677 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.751 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:28.183 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.752 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.753 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.753 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.754 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.754 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.755 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.755 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:31.784 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.756 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:44:32.335 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.757 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.757 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.758 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.758 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.759 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.760 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.760 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:35.928 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.761 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:36.482 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:36.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:02.761 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.762 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.762 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.763 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.764 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.764 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.765 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:40.074 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.765 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:40.640 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:40.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:02.766 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.767 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.767 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.768 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.768 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.769 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.769 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:44.216 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.770 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:44.769 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:44.770 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:02.771 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.771 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.772 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.772 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.773 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.774 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.774 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:44:48.354 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.775 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:48.922 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:48.923 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:02.775 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.776 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.776 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.777 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.778 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.778 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.779 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:52.795 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.779 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:53.346 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:53.347 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:02.780 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.780 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.781 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.782 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.782 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.783 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.783 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:44:56.919 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.784 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:44:57.483 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:44:57.484 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:02.784 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.785 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.786 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.786 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.787 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.787 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.788 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.789 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:45:01.620 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:01.621 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:02.789 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:02.188 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:02.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:45:02.191 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.790 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.790 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.791 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.792 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.792 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.793 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.793 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:45:05.769 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:05.770 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:02.794 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:06.319 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:06.320 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:45:06.352 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.795 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.795 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.796 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.796 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.797 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.798 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.798 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:45:09.909 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:09.910 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:02.799 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:10.477 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:10.478 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:45:10.515 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.799 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.800 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.801 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.801 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.802 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.802 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.803 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.803 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:14.613 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:14.614 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:45:14.677 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.804 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:15.183 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:45:15.183 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.805 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.805 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.806 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.806 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.807 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.808 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:45:18.450 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:18.451 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:02.808 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:19.019 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:19.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:45:19.022 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.809 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.809 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.810 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.811 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.811 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.812 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.813 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:45:22.594 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:22.595 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2022-02-17T21:01:02.813 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:23.148 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:23.150 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:45:23.175 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.814 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.814 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.815 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.816 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.816 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.817 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.817 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:45:26.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:26.749 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2022-02-17T21:01:02.818 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:27.300 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:27.301 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:45:27.334 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.818 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.819 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.820 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.820 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.821 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.821 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.822 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.823 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:31.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:31.442 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:45:31.492 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.823 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:31.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:45:31.999 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.824 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.824 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.825 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.826 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.826 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.827 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.827 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:35.574 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:45:35.574 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:45:35.652 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.828 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:36.138 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:45:36.159 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.828 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.829 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.830 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.830 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.831 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.831 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.832 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:45:39.806 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.833 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:40.256 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:45:40.314 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.833 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.834 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.834 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.835 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.836 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.836 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.837 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.837 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:44.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:45:44.473 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.838 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:44.982 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.838 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.839 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.840 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.840 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.841 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.841 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.842 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:48.727 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:45:48.826 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.842 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:49.333 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.843 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.844 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.844 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.845 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.845 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.846 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.847 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:45:52.979 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.847 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:53.489 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.848 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.848 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.849 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.849 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.850 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.851 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.851 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:45:57.111 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.852 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:45:57.648 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.852 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.853 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.853 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.854 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.855 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.855 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.856 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:01.253 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.856 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:01.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:01.806 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:02.857 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.858 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.858 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.859 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.859 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.860 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.860 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:05.381 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.861 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:05.947 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:05.948 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:02.862 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.862 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.863 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.863 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.864 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.865 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.865 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:09.527 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.866 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:10.078 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:10.079 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:02.866 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.867 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.868 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.868 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.869 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.869 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.870 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.871 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:14.453 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:14.454 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:02.871 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:15.003 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:15.004 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:46:15.057 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.872 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.872 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.873 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.874 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.874 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.875 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.875 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.876 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:19.147 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:19.148 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:46:19.222 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.876 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:19.698 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:46:19.730 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.877 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.878 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.878 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.879 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.879 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.880 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.880 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.881 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:23.829 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:46:23.888 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.882 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:24.395 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.882 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.883 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.883 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.884 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.884 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.885 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.886 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:27.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:46:28.043 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.886 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:28.550 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.887 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.887 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.888 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.888 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.889 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.890 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:46:31.641 DEBUG:teuthology.orchestra.run:got remote process result: 124 2022-02-17T21:01:02.890 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:32.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:32.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:02.891 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:32.660 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:32.661 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:46:32.707 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.891 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.892 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.893 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.893 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.894 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.894 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.895 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.895 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:36.799 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:36.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:02.896 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:37.350 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:37.350 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:46:37.375 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.897 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.897 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.898 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.898 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.899 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.900 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.900 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:46:41.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:41.021 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:02.901 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:41.588 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:41.590 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:46:41.625 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.901 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.902 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.903 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.903 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.904 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.904 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.905 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.905 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:45.719 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:45.720 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:46:45.784 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.906 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:46.287 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:46:46.291 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.907 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.907 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.908 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.908 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.909 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.909 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.910 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:49.900 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:49.901 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:46:49.941 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.911 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.911 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.912 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.912 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.913 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.914 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.914 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.915 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:54.034 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:54.035 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:46:54.102 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.915 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:54.601 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:46:54.608 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.916 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.917 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.917 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.918 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.918 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.919 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.920 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:58.173 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:46:58.174 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:46:58.260 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.920 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:46:58.727 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:46:58.768 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.921 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.921 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.922 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.922 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.923 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.923 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.924 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.925 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:02.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:47:02.928 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.925 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:03.612 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.926 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.926 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.927 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.927 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.928 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.929 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.929 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:47:07.219 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.930 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:07.774 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.930 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.931 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.931 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.932 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.933 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.933 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.934 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:47:11.360 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.934 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:47:11.910 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:47:11.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:02.935 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.936 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.936 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.937 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.937 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.938 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.939 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:47:15.508 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.939 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:47:16.059 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:47:16.060 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:02.940 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.940 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.941 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.941 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.942 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.943 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.943 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.944 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:47:20.207 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:47:20.209 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:02.944 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.945 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.946 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.946 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.947 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.947 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.948 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.948 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:47:24.336 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:47:24.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2022-02-17T21:01:02.949 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:24.898 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:47:24.898 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:47:24.907 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.950 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:02.950 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.951 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:02.951 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.952 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.952 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.953 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:47:28.481 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:47:28.481 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:02.954 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:29.032 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:47:29.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:47:29.106 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.954 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:29.606 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:47:29.612 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.955 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.955 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.956 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.957 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.957 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.958 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.958 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:33.176 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:47:33.177 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:47:33.262 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.959 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:33.732 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:47:33.770 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.959 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.960 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.961 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.961 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.962 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:02.962 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.963 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:37.316 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:47:37.317 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:47:37.414 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.963 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:37.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:47:37.923 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.964 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:02.965 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.965 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:02.966 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.966 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.967 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.967 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:47:41.569 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:02.968 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:42.014 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:47:42.077 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.969 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:42.584 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.969 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.970 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.970 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.971 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.972 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.972 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.973 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:46.163 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:47:46.238 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.973 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:46.745 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.974 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.975 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.975 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.976 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.976 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:02.977 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.977 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:47:50.394 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.978 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:50.901 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.979 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.979 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.980 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.980 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.981 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.981 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.982 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:54.673 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:47:54.759 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.983 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:55.266 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.983 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.984 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.984 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:02.985 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.986 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:02.986 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.987 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:58.821 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:47:58.915 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.987 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:47:59.424 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.988 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:02.988 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.989 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.990 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.990 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.991 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.991 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:48:03.073 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:02.992 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:03.580 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.992 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.993 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.994 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.994 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:02.995 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:02.995 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:02.996 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:07.206 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:02.997 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:07.738 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:02.997 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:02.998 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:02.998 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:02.999 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.000 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.000 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.001 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:11.332 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.001 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:11.883 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:48:11.883 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:03.002 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.002 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.003 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.004 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.004 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.005 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.005 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:15.452 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.006 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:16.021 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:48:16.022 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:03.007 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.007 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.008 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.008 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.009 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.009 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.010 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.011 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:20.295 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.011 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.012 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.012 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.013 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.014 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.014 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.015 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:23.903 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.015 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:24.449 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.016 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.016 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.017 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.018 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.018 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.019 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.019 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:28.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.020 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:28.603 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:48:28.604 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2022-02-17T21:01:03.020 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.021 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.022 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.022 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.023 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.023 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.024 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:32.174 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.025 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:32.739 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:48:32.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:03.025 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.026 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:33.785 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.026 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.027 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.027 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.028 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.029 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.029 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.030 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:37.338 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:48:37.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:48:37.437 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.030 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:37.892 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:48:37.944 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.031 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.032 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.032 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.033 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.033 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.034 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.034 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.035 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:42.116 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:48:42.181 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.036 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:42.689 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.036 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.037 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.037 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.038 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.039 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.039 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.040 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:46.255 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:48:46.341 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.040 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:46.848 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.041 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.042 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.042 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.043 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.043 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.044 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.044 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:50.400 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:48:50.499 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.045 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:51.006 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.046 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.046 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.047 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.047 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.048 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.049 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.049 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:54.658 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.050 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:55.167 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.050 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.051 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.051 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.052 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.053 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.053 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.054 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:48:58.783 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.054 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:48:59.325 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.055 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.056 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.057 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.057 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.058 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.059 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.059 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:49:03.041 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.060 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:03.553 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.060 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.061 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.062 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.062 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.063 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.063 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.064 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:49:07.157 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.064 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:07.712 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.065 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.066 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.066 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.067 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.067 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.068 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.069 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:49:11.286 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.069 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:49:11.835 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:11.836 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:03.070 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.071 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.071 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.072 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.072 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.073 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.073 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.074 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:49:15.967 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:15.968 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:03.075 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:16.517 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:16.518 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:49:16.531 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.075 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.076 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.076 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.077 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.078 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.078 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.079 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:49:20.090 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:20.091 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:03.079 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:20.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:20.666 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:49:20.684 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.080 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.081 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.081 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.082 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.083 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.083 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.084 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:49:24.234 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:24.235 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:03.084 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:24.787 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:24.787 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:49:24.842 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.085 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.085 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.086 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.087 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.087 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.088 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.088 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.089 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:28.934 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:28.935 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:49:29.000 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.090 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:29.487 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:49:29.506 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.090 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.091 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.091 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.092 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.093 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.093 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.094 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:33.207 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:33.208 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:49:33.268 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.094 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:33.772 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:49:33.776 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.095 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.096 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.096 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.097 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.097 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.098 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.099 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:37.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:49:37.345 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:49:37.430 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.099 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:37.898 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:49:37.937 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.100 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.100 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.101 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.101 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.102 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.103 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.103 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:49:41.590 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.104 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:42.050 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:49:42.096 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.104 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:42.603 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.105 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.106 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.107 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.107 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.108 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.109 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.109 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:46.175 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:49:46.254 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.110 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:46.761 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.111 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.111 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.112 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.113 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.113 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.114 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.114 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:50.324 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:49:50.411 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.115 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:50.918 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.116 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.117 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.117 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.118 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.119 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.119 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.120 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:49:54.562 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.120 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:55.076 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.121 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.122 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.123 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.124 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.125 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.126 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.127 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:49:58.703 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.127 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:49:59.236 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.128 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.129 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.129 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.130 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.130 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.131 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.132 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:02.930 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.132 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:03.482 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:03.483 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2022-02-17T21:01:03.133 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.134 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.135 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.135 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.136 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.137 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.137 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:07.062 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.138 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:07.634 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:07.635 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2022-02-17T21:01:03.139 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.139 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.140 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.140 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.141 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.142 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.142 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.143 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:11.758 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:11.759 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:03.144 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.144 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.145 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.146 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.146 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.147 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.147 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.148 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:15.882 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:15.883 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:03.149 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:16.449 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:16.450 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:50:16.478 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.149 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.150 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.152 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.152 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.153 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.154 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.154 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:50:20.028 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:20.029 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:03.155 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:20.578 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:20.579 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:50:20.635 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.156 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.156 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.157 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.158 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.158 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.159 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.159 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.160 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:24.714 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:24.715 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:50:24.790 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.161 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:25.265 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:50:25.300 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.162 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.162 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.163 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.163 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.164 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.165 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.165 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:28.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:28.858 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:50:28.950 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.166 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:29.408 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:50:29.457 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.167 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:29.963 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.167 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.168 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.168 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.169 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.170 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.171 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.171 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:33.666 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:50:33.709 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.172 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:34.217 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.173 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.173 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.174 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.174 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.175 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.176 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.176 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:37.786 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.177 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:38.357 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:38.358 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:03.178 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.179 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.179 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.180 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.180 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.181 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.182 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:41.936 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.182 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:42.489 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:42.490 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2022-02-17T21:01:03.183 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.184 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.184 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.185 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.185 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.186 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.187 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.188 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:46.621 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:46.622 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:03.188 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:47.180 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:47.181 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:50:47.195 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.189 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.189 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.190 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.191 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.191 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.192 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.193 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:50:50.753 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:50.754 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:03.193 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:51.321 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:51.322 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:50:51.352 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.194 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.194 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.195 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.196 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.197 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.197 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.198 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.199 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:55.449 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:55.450 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:50:55.511 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.199 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:56.013 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:50:56.018 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.200 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.200 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.201 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.202 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.202 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.203 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.204 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:50:59.606 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:50:59.607 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:50:59.668 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.204 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:00.157 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:51:00.176 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.205 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.206 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.206 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.207 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.208 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.208 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.209 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:03.734 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:51:03.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:51:03.825 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.210 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:04.330 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:51:04.376 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.210 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:04.882 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.211 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.211 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.212 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.213 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.213 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.214 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.215 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:08.449 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:51:08.530 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.216 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:09.037 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.216 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.217 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.217 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.218 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.219 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.219 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.220 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:12.596 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:51:12.690 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.220 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:13.195 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.221 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.222 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.222 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.223 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.224 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.225 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.225 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:51:16.833 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.226 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:17.351 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.226 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.227 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.228 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.228 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.229 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.230 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.230 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:51:20.962 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.231 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:21.506 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.231 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.232 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.233 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.234 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.235 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.235 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.236 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:51:25.096 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.236 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:51:25.663 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:51:25.664 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:03.237 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.238 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.238 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.239 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.240 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.240 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.241 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:51:29.244 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.242 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:51:29.799 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:51:29.800 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:03.243 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.243 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.244 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.245 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.245 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.246 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.247 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.247 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:51:33.931 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:51:33.932 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:03.248 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:34.659 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.249 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.249 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.250 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.251 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.251 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.252 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.253 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:51:38.239 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:51:38.240 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2022-02-17T21:01:03.253 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:38.808 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:51:38.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:51:38.819 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.254 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.255 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.255 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.256 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.257 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.257 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.258 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:51:42.375 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:51:42.376 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2022-02-17T21:01:03.258 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:42.923 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:51:42.924 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:51:42.973 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.259 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.260 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.260 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.261 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.262 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.262 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.263 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.263 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:47.052 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:51:47.053 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:51:47.131 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.264 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:47.603 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:51:47.638 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.265 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.265 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.266 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.267 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.267 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.268 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.268 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:51:51.286 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.269 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:51.734 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:51:51.794 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.270 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:52.302 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.270 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.271 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.272 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.272 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.273 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.273 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.274 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:55.868 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:51:55.953 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.275 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:51:56.460 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.275 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.276 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.277 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.277 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.278 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.279 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.279 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:00.097 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.280 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:00.618 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.280 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.281 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.282 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.282 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.283 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.283 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.284 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:04.245 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.285 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:04.775 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.285 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.286 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.287 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.288 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.288 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.289 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.289 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:08.431 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.290 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:08.969 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.291 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.291 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.292 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.293 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.293 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.294 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.294 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:12.552 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.295 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:13.101 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:13.102 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:03.296 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.297 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.297 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.298 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.299 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.299 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.300 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.300 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:17.239 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:17.240 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:03.301 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.302 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.302 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.303 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.304 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.304 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.305 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.306 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:21.354 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:21.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:03.306 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:21.924 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:21.924 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:52:21.941 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.307 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.307 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.308 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.309 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.309 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.310 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.311 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:25.494 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:25.495 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:03.311 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:26.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:26.048 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:52:26.095 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.312 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.312 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.313 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.314 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.314 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.315 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.316 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.316 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:30.180 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:30.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:52:30.253 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.317 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:30.734 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:52:30.762 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.317 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.318 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.319 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.319 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.320 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.321 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.321 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.322 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:34.860 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:52:34.918 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.323 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:35.425 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.323 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.324 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.324 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.325 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.326 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.327 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.327 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.328 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:39.644 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:39.645 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:52:39.715 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.329 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.330 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.330 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.331 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.332 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.333 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.333 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:43.269 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.334 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:43.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:43.820 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:03.335 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.336 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.336 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.337 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.338 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.338 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.343 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.344 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:52:47.959 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:47.960 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:03.345 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:48.509 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:48.510 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:52:48.538 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.345 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.346 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.347 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.347 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.348 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.349 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.349 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.350 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:52.641 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:52.642 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:52:52.692 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.351 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:53.191 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:52:53.200 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.351 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.352 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.353 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.353 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.354 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.355 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.356 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:56.768 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:52:56.769 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:52:56.852 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.357 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:52:57.339 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:52:57.360 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.358 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.359 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.359 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.360 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.361 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.362 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.363 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:00.916 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:53:00.917 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:53:01.007 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.364 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:01.472 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:53:01.514 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.365 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.366 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.367 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.368 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.369 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.370 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.371 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:05.175 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:53:05.175 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:53:05.249 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.372 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:05.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:53:05.758 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.373 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.375 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.375 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.376 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.377 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.378 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.379 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:09.323 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:53:09.323 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:53:09.411 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.380 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:09.873 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:53:09.920 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.381 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.382 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.383 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.384 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.385 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.386 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.386 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:53:13.573 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.387 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:14.021 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:53:14.079 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.387 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:14.586 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.388 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.389 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.389 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.390 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.390 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.391 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.392 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:18.146 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:53:18.236 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.392 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:18.743 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.393 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.394 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.394 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.395 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.395 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.396 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.397 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:22.300 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:53:22.394 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.397 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:22.900 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.398 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.398 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.399 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.400 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.400 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.401 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.402 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:53:26.547 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.402 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:27.060 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.403 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.403 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.404 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.405 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.405 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.406 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.406 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:53:30.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.407 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:31.218 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.408 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.408 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.409 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.410 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.410 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.411 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.411 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:34.896 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:53:34.955 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.412 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:35.462 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.413 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.413 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.414 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.415 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.415 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.416 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.416 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:39.024 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops , 2022-02-17T10:53:39.112 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.417 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:39.619 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.418 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.418 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.419 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.420 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.420 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.421 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.421 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:53:43.267 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.422 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:43.775 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.423 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.423 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.424 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.425 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.426 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.426 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.427 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:53:47.411 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.428 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:47.933 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.428 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.429 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.430 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.431 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.431 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.432 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.433 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:53:51.560 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.433 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:53:52.093 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.434 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.435 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.435 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.436 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.436 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.437 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.438 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:53:55.679 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.438 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:53:56.248 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:53:56.249 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:03.439 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.440 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.440 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.441 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.442 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.442 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.443 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:53:59.822 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.444 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:54:00.375 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:54:00.376 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:03.444 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.445 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.446 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.446 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.447 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.448 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.448 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.449 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:54:04.556 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:54:04.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:03.449 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:05.108 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:54:05.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:54:05.130 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.450 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.451 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.451 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.452 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.453 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.453 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.454 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:54:08.688 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:54:08.689 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2022-02-17T21:01:03.455 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:09.252 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:54:09.253 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:54:09.294 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.456 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.457 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.458 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.459 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.460 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.461 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.461 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.462 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:13.373 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:54:13.373 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:54:13.453 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.463 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:13.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:54:13.960 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.464 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.464 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.465 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.466 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.467 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.468 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.468 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:54:17.607 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.469 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:18.053 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T10:54:18.114 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.470 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:18.622 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.470 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.471 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.471 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.472 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.473 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.473 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.474 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:22.201 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:54:22.274 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.474 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:22.782 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.475 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.476 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.476 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.477 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.478 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.478 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.479 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:26.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:54:26.435 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.479 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:26.942 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.480 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.481 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.481 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.482 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.482 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.483 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.484 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:54:30.755 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.484 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:31.267 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.485 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.486 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.486 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.487 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.487 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.488 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.489 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:54:34.891 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.489 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:35.420 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.490 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.490 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.491 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.492 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.492 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.493 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.493 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:54:39.028 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.494 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:39.576 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.495 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:40.074 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:54:40.084 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.495 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.496 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.496 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.497 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.498 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.498 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.499 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.500 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:44.181 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:54:44.182 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:54:44.241 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.500 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:44.746 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:54:44.747 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.501 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.501 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.502 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.503 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.503 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.504 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.504 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:48.324 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:54:48.324 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:54:48.396 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.505 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:48.874 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:54:48.902 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.506 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.506 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.507 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.507 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.508 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.509 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.509 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:52.458 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:54:52.459 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:54:52.551 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.510 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:53.020 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:54:53.058 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.510 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.511 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.512 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.512 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.513 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.514 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.514 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.515 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:54:57.372 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.515 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:54:57.886 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.516 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.517 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.517 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.518 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.519 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.519 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.520 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:01.509 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.520 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:02.046 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.521 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.522 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.522 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.523 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.524 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.524 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.525 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:05.624 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.525 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:06.190 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:55:06.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2022-02-17T21:01:03.526 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.527 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.527 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.528 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.528 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.529 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.530 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:09.759 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.530 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:10.308 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:55:10.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:03.531 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.531 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.532 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.533 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.533 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.534 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.535 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.535 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:14.453 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:55:14.454 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:03.536 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:15.005 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:55:15.006 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2022-02-17T10:55:15.023 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.536 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.537 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.538 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.538 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.539 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.539 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.540 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:18.605 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:55:18.606 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:03.541 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:19.166 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:55:19.167 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:55:19.183 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.541 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.542 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.542 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.543 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.544 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.544 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.545 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.546 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:23.378 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:55:23.379 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:55:23.457 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.546 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:23.929 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2022-02-17T10:55:23.968 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.547 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.548 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.548 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.549 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.550 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.550 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.551 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.551 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:28.079 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:55:28.121 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.552 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.553 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.553 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.554 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.554 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.555 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.556 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.556 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:32.198 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:55:32.280 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.557 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:32.788 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.557 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.558 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.559 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.559 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.560 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.560 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.561 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:55:36.438 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.561 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:36.945 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.562 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.563 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.563 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.564 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.564 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.565 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.566 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:40.610 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:55:40.692 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.566 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:41.199 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.567 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.567 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.568 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.569 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.569 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.570 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.571 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:44.753 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:55:44.849 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.571 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:45.355 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.572 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.572 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.573 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.574 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.574 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.575 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.575 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:48.987 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.576 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:49.512 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.577 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.577 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.578 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.578 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.579 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.580 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.580 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:53.124 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.581 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:55:53.664 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.581 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.582 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.583 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.583 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.584 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.585 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.585 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:57.236 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.586 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:55:57.803 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:55:57.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:03.586 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.587 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.588 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.588 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.589 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.589 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.590 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:01.375 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.591 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:01.925 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:56:01.926 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:03.591 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:02.488 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:56:02.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:56:02.490 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.592 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.592 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.593 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.594 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.594 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.595 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.595 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:06.249 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:56:06.250 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:03.596 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:06.800 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:56:06.801 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:56:06.839 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.597 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.597 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.598 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.598 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.599 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.600 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.600 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.601 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:10.933 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:56:10.934 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:56:10.999 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.601 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:11.483 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:56:11.505 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.602 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.603 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.603 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.604 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.605 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.605 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.606 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:56:15.158 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.606 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:15.624 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:56:15.666 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.607 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.608 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.608 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.609 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.609 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.610 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.611 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.611 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:19.759 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:56:19.825 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.612 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:20.332 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.613 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.613 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.614 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.614 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.615 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.615 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.616 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:23.897 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:56:23.982 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.617 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:24.489 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.617 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.618 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.619 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.619 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.620 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.620 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.621 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:28.132 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.621 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:28.644 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.622 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.623 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.624 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.624 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.625 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.625 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.626 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:32.288 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.627 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:32.817 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.627 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.628 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.629 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.629 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.630 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.631 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.631 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:36.428 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.632 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:36.977 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:56:36.978 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:03.633 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.633 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.634 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.634 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.635 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.636 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.636 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:40.549 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.637 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:41.115 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:56:41.116 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:03.637 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.638 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:42.147 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.639 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.639 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.640 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.640 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.641 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.642 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.642 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.643 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:46.229 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:56:46.306 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.643 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:46.813 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.644 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.645 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.645 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.646 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.646 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.647 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.648 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:50.379 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops , 2022-02-17T10:56:50.461 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.648 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:50.968 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.649 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.649 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.650 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.651 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.651 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.652 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.652 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:56:54.618 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.653 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:55.126 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.653 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.654 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.655 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.655 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.656 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.656 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.657 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:56:58.759 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.658 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:56:59.285 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.658 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.659 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.660 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.660 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.661 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.661 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.662 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:02.962 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.662 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:03.518 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.663 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.664 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.664 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.665 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.666 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.667 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.668 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:07.111 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.668 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:07.661 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:07.662 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:03.669 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.669 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.670 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.671 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.672 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.672 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.673 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:11.240 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.673 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:11.807 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:11.808 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:03.674 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.675 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.675 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.676 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.676 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.677 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.678 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.678 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:15.917 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:15.918 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2022-02-17T21:01:03.679 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:16.487 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:16.488 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:57:16.499 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.679 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.680 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.681 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.681 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.682 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.682 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.683 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:20.070 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:20.071 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2022-02-17T21:01:03.684 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:20.623 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:20.624 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:57:20.656 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.684 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.685 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.686 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.686 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.687 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.688 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.689 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.690 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:24.762 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:24.763 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T10:57:24.815 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.690 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:25.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:57:25.321 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.691 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.692 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.693 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.694 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.694 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2022-02-17T10:57:27.940 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.695 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:28.509 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:28.510 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2022-02-17T21:01:03.696 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.696 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.697 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.698 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.698 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.699 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.700 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.700 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:32.633 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:32.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2022-02-17T21:01:03.701 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:33.208 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:33.208 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2022-02-17T10:57:33.215 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.702 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.702 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.703 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.704 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.704 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.705 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.706 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:57:36.773 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:36.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2022-02-17T21:01:03.706 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:37.325 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:37.326 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:57:37.372 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.707 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.708 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.708 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.709 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.710 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.710 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.711 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.712 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:41.467 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:41.468 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:57:41.531 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.712 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:42.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2022-02-17T10:57:42.037 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.713 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.714 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.714 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.715 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.715 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.716 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.717 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:45.588 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:57:45.589 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:57:45.682 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.718 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:46.154 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight , 2022-02-17T10:57:46.191 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.719 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.719 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.720 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.721 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.722 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.722 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.723 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.723 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:50.281 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:57:50.352 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.724 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:50.858 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.725 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.725 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.726 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.727 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.728 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.729 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.729 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:54.419 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:57:54.508 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.730 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:55.016 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.731 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.731 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.732 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.733 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.734 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.735 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.735 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:58.593 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:57:58.693 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.736 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:57:59.201 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.737 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.738 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.739 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.739 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.740 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.741 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.741 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:02.853 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.742 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:03.359 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.743 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.744 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.744 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.745 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.746 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.747 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.748 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:06.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.748 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:07.514 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.749 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.750 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.750 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.751 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.752 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.752 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.753 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:11.121 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.753 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:11.670 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:11.672 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2022-02-17T21:01:03.754 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.755 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.756 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.756 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.757 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.758 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.759 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:15.266 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.759 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:15.820 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:15.820 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2022-02-17T21:01:03.760 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.761 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.761 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.762 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.763 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.764 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.764 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:19.386 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.765 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:19.952 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:19.953 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2022-02-17T21:01:03.766 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.766 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.767 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.768 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.768 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.769 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.770 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.770 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:24.073 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:24.074 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2022-02-17T21:01:03.771 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:24.710 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:24.711 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:58:24.750 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.771 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.772 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.773 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.773 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.774 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.775 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.776 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.776 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:28.828 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:28.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:58:28.907 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.777 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:29.396 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops , 2022-02-17T10:58:29.415 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.778 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.778 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.779 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.780 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.780 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.781 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.781 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:32.965 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:32.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:58:33.064 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.782 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:33.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2022-02-17T10:58:33.571 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.783 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.783 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.784 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.785 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.786 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.786 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.787 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.788 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:37.652 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:58:37.732 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.788 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:38.237 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.789 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.789 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.790 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.791 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.792 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.792 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.793 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:58:41.882 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.793 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:42.389 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.794 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.795 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.796 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.796 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.797 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.798 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.799 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.799 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:58:46.481 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:46.482 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:03.800 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:47.044 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:47.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:58:47.054 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.801 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.802 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.802 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.803 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.803 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.804 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.805 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.805 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:51.287 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:51.288 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:58:51.359 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.806 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:51.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T10:58:51.866 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.806 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.807 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.808 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.809 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.809 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.810 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.811 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:55.422 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:58:55.423 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:58:55.515 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.812 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:58:55.974 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T10:58:56.021 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.813 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.813 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.814 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.815 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.816 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.816 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.817 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.818 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:00.113 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T10:59:00.177 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.819 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:00.685 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.819 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.820 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.821 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.822 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.822 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.823 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.824 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:04.250 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T10:59:04.337 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.824 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:04.845 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.825 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.826 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.827 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.828 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.828 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.829 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.830 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:59:08.498 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.831 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:09.005 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.831 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.832 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.833 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.834 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.834 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.835 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.836 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:59:12.619 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.837 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:13.166 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.837 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.838 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.839 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.840 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.840 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.841 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.842 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:59:16.906 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.843 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:59:17.455 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:59:17.456 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:03.843 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.844 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.845 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.846 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.846 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.847 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.848 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.848 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:59:21.596 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:59:21.597 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:03.849 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:22.147 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:59:22.147 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T10:59:22.149 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.850 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.851 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.851 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.852 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.853 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.853 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.854 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:59:25.725 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:59:25.725 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2022-02-17T21:01:03.855 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:26.292 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:59:26.293 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2022-02-17T10:59:26.308 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.855 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.856 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.857 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.858 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.858 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.859 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.860 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:59:29.863 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:59:29.864 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2022-02-17T21:01:03.861 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:30.414 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:59:30.414 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:59:30.469 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.861 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.862 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.863 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.864 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.864 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.865 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.866 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.867 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:34.564 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:59:34.564 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:59:34.634 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.867 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:35.117 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight , 2022-02-17T10:59:35.139 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.868 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.869 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.870 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.870 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.871 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.872 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.872 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:38.695 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T10:59:38.696 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:59:38.789 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.873 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:39.263 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops , 2022-02-17T10:59:39.298 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.874 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.874 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.875 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.876 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.876 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.877 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.877 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.878 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:43.428 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2022-02-17T10:59:43.493 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.879 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:44.000 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.879 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.880 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.880 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.881 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.882 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.883 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.883 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:47.572 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2022-02-17T10:59:47.651 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.884 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:48.158 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.885 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.886 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.887 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.887 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.888 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.889 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.890 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , 2022-02-17T10:59:51.808 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.891 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:52.315 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.891 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.892 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.892 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.893 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.894 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.894 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.895 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T10:59:55.942 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.896 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T10:59:56.471 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.896 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.897 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.897 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.898 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.899 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.899 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.900 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:00.076 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.901 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:00.628 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.901 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.902 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.903 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.903 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.904 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.905 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.906 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:04.195 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.907 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:04.758 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:04.759 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2022-02-17T21:01:03.907 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.908 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.909 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.909 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.910 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.911 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.911 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.912 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:08.985 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:08.985 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2022-02-17T21:01:03.912 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:09.548 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:09.549 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2022-02-17T11:00:09.577 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.913 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.914 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.914 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.915 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.915 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.916 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.917 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:13.138 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:13.139 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2022-02-17T21:01:03.917 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:13.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:13.690 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T11:00:13.738 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.918 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.918 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.919 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.920 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.921 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.921 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.922 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.923 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:17.837 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:17.838 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T11:00:17.900 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.924 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:18.390 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2022-02-17T11:00:18.411 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.924 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.925 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.925 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.926 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.927 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.927 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.928 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:21.969 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:21.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T11:00:22.060 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.928 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:22.543 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2022-02-17T11:00:22.567 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.929 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.930 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.930 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.931 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.931 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.932 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.933 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:26.120 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:26.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T11:00:26.219 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.933 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:26.673 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2022-02-17T11:00:26.728 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.934 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.934 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.935 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.936 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.936 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.937 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.938 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.938 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:30.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2022-02-17T11:00:30.886 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.939 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:31.395 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.939 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.940 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.941 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.942 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.942 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.943 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.944 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:35.266 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.945 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:35.822 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.945 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.946 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.946 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.947 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.948 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.948 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.949 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:39.401 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.949 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:39.954 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:39.955 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2022-02-17T21:01:03.950 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.950 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.951 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.952 WARNING:scrape:Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) 2022-02-17T21:01:03.952 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall 2022-02-17T21:01:03.953 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send 2022-02-17T21:01:03.953 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.954 WARNING:scrape:Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:44.092 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:44.093 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2022-02-17T21:01:03.955 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:44.641 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2022-02-17T11:00:44.641 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2022-02-17T11:00:44.647 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.955 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write 2022-02-17T21:01:03.956 WARNING:scrape:Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) 2022-02-17T21:01:03.957 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.957 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.958 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.959 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.960 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed 2022-02-17T21:01:03.960 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.961 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:49.232 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2022-02-17T11:00:49.316 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): 2022-02-17T21:01:03.962 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:49.823 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.962 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all 2022-02-17T21:01:03.963 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.964 WARNING:scrape:Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) 2022-02-17T21:01:03.964 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.965 WARNING:scrape:Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") 2022-02-17T21:01:03.965 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.966 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:53.472 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None 2022-02-17T21:01:03.966 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:53.980 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.967 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.968 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.968 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.969 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.969 WARNING:scrape:Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) 2022-02-17T21:01:03.970 WARNING:scrape:Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2022-02-17T21:01:03.971 WARNING:scrape:Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2022-02-17T11:00:57.606 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-02-17T21:01:03.971 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:00:58.144 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal 2022-02-17T21:01:03.972 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) 2022-02-17T21:01:03.972 WARNING:scrape:Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) 2022-02-17T21:01:03.973 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write 2022-02-17T21:01:03.973 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send 2022-02-17T21:01:03.974 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/contextutil.py", line 133, in __call__ , raise MaxWhileTries(error_msg) , teuthology.exceptions.MaxWhileTries: reached maximum tries (500) after waiting for 3000 seconds 2022-02-17T21:01:03.975 WARNING:scrape:Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/contextutil.py", line 33, in nested , yield vars 2022-02-17T21:01:03.975 WARNING:scrape:Ignoring malparsed backtrace: return self.controller.run(**kwargs) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/orchestra/remote.py", line 509, in run , r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2022-02-17T21:01:03.976 WARNING:scrape:Ignoring malparsed backtrace: suppress = manager.__exit__(*exc_info) , File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ , next(self.gen) 2022-02-17T21:01:03.976 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:11:44.258 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 44 -267us[ -267us] +/- 72ms , 2022-02-17T11:11:44.259 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample , 2022-02-17T11:11:44.259 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2022-02-17T21:01:03.977 WARNING:scrape:Ignoring malparsed backtrace: 2022-02-17T11:11:54.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.base , 2022-02-17T11:11:54.903 INFO:teuthology.task.internal:Tidying up after the test... , 2022-02-17T11:11:54.904 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-02-17T21:01:04.187 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/results.py", line 34, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/results.py", line 69, in results Scraper(archive_dir).analyze() File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/scrape.py", line 490, in analyze if reason.match(job): File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/scrape.py", line 75, in match if (self.backtrace is None) != (job.get_backtrace() is None): File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/scrape.py", line 338, in get_backtrace self._populate_backtrace() File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/scrape.py", line 384, in _populate_backtrace bt, ass = self._search_backtrace(GzipFile(gzipped_log_path)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/teuthology/scrape.py", line 304, in _search_backtrace if ".stderr:" in line: TypeError: a bytes-like object is required, not 'str'