Found 427 jobs Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f8c2c4c8b20] , 2: gsignal() Ignoring malparsed backtrace: for result in self: , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/parallel.py", line 98, in __next__ , resurrect_traceback(result) Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f8c2c4c8b20] , 2: gsignal() Ignoring malparsed backtrace: for result in self: , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/parallel.py", line 98, in __next__ , resurrect_traceback(result) Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f8c2c4c8b20] , 2: gsignal() Ignoring malparsed backtrace: for result in self: , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/parallel.py", line 98, in __next__ , resurrect_traceback(result) Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f8c2c4c8b20] , 2: gsignal() Ignoring malparsed backtrace: for result in self: , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/parallel.py", line 98, in __next__ , resurrect_traceback(result) Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f8c2c4c8b20] , 2: gsignal() Ignoring malparsed backtrace: for result in self: , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/parallel.py", line 98, in __next__ , resurrect_traceback(result) Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f8c2c4c8b20] , 2: gsignal() Ignoring malparsed backtrace: for result in self: , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/parallel.py", line 98, in __next__ , resurrect_traceback(result) Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f8c2c4c8b20] , 2: gsignal() Ignoring malparsed backtrace: for result in self: , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/parallel.py", line 98, in __next__ , resurrect_traceback(result) Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0435d02b20] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T22:09:26.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo , 2021-11-23T22:09:26.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... , 2021-11-23T22:09:26.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0435d02b20] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T22:09:26.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo , 2021-11-23T22:09:26.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... , 2021-11-23T22:09:26.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0435d02b20] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T22:09:26.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo , 2021-11-23T22:09:26.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... , 2021-11-23T22:09:26.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0435d02b20] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T22:09:26.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo , 2021-11-23T22:09:26.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... , 2021-11-23T22:09:26.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0435d02b20] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T22:09:26.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo , 2021-11-23T22:09:26.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... , 2021-11-23T22:09:26.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0435d02b20] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T22:09:26.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo , 2021-11-23T22:09:26.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... , 2021-11-23T22:09:26.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0435d02b20] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T22:09:26.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo , 2021-11-23T22:09:26.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... , 2021-11-23T22:09:26.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0435d02b20] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T22:09:26.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo , 2021-11-23T22:09:26.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... , 2021-11-23T22:09:26.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0435d02b20] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T22:09:26.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo , 2021-11-23T22:09:26.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... , 2021-11-23T22:09:26.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers Ignoring malparsed backtrace: ceph version 17.0.0-9192-g64d7747e (64d7747ea0581787abfa8e65c90131fc6a29c90f) quincy (dev) , 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7f29eb8623c0] , 2: gsignal() Ignoring malparsed backtrace: 2021-11-23T20:24:53.747 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 1.9652000000000001e-05 , 2021-11-23T20:24:53.747 INFO:teuthology.orchestra.run.smithi124.stdout: }, , 2021-11-23T20:24:53.747 INFO:teuthology.orchestra.run.smithi124.stdout: { Ignoring malparsed backtrace: 2021-11-23T20:24:53.769 INFO:teuthology.orchestra.run.smithi124.stdout: "tid": 62425 , 2021-11-23T20:24:53.769 INFO:teuthology.orchestra.run.smithi124.stdout: }, , 2021-11-23T20:24:53.769 INFO:teuthology.orchestra.run.smithi124.stdout: "events": [ Ignoring malparsed backtrace: 2021-11-23T20:24:53.790 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:30.745052+0000", , 2021-11-23T20:24:53.790 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 5.3254000000000002e-05 , 2021-11-23T20:24:53.790 INFO:teuthology.orchestra.run.smithi124.stdout: }, Ignoring malparsed backtrace: 2021-11-23T20:24:53.811 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:30.752137+0000", , 2021-11-23T20:24:53.812 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 0.000158672 , 2021-11-23T20:24:53.812 INFO:teuthology.orchestra.run.smithi124.stdout: }, Ignoring malparsed backtrace: 2021-11-23T20:24:53.833 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:30.794905+0000", , 2021-11-23T20:24:53.833 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 4.3988e-05 , 2021-11-23T20:24:53.833 INFO:teuthology.orchestra.run.smithi124.stdout: }, Ignoring malparsed backtrace: 2021-11-23T20:24:53.854 INFO:teuthology.orchestra.run.smithi124.stdout: "client_addr": "v1:172.21.15.103:0/3887916508", , 2021-11-23T20:24:53.854 INFO:teuthology.orchestra.run.smithi124.stdout: "tid": 62631 , 2021-11-23T20:24:53.854 INFO:teuthology.orchestra.run.smithi124.stdout: }, Ignoring malparsed backtrace: 2021-11-23T20:24:53.876 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "all_read", , 2021-11-23T20:24:53.876 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:30.862474+0000", , 2021-11-23T20:24:53.876 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 0.00010130500000000001 Ignoring malparsed backtrace: 2021-11-23T20:24:53.897 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "started", , 2021-11-23T20:24:53.898 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:30.940721+0000", , 2021-11-23T20:24:53.898 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 0.00013864699999999999 Ignoring malparsed backtrace: 2021-11-23T20:24:53.919 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "commit_sent", , 2021-11-23T20:24:53.919 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:31.169807+0000", , 2021-11-23T20:24:53.919 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 5.4113000000000002e-05 Ignoring malparsed backtrace: 2021-11-23T20:24:53.940 INFO:teuthology.orchestra.run.smithi124.stdout: "client": "client.4491", , 2021-11-23T20:24:53.940 INFO:teuthology.orchestra.run.smithi124.stdout: "client_addr": "v1:172.21.15.103:0/3887916508", , 2021-11-23T20:24:53.940 INFO:teuthology.orchestra.run.smithi124.stdout: "tid": 63704 Ignoring malparsed backtrace: 2021-11-23T20:24:53.961 INFO:teuthology.orchestra.run.smithi124.stdout: { , 2021-11-23T20:24:53.961 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "all_read", , 2021-11-23T20:24:53.962 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:31.981557+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:53.983 INFO:teuthology.orchestra.run.smithi124.stdout: { , 2021-11-23T20:24:53.983 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "started", , 2021-11-23T20:24:53.983 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:32.013736+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:54.004 INFO:teuthology.orchestra.run.smithi124.stdout: { , 2021-11-23T20:24:54.004 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "commit_sent", , 2021-11-23T20:24:54.004 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:32.051307+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:54.034 INFO:teuthology.orchestra.run.smithi124.stdout: }, , 2021-11-23T20:24:54.035 INFO:teuthology.orchestra.run.smithi124.stdout: { , 2021-11-23T20:24:54.035 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "dispatched", Ignoring malparsed backtrace: 2021-11-23T20:24:54.063 INFO:teuthology.orchestra.run.smithi124.stdout: "flag_point": "commit sent; apply or cleanup", , 2021-11-23T20:24:54.063 INFO:teuthology.orchestra.run.smithi124.stdout: "events": [ , 2021-11-23T20:24:54.063 INFO:teuthology.orchestra.run.smithi124.stdout: { Ignoring malparsed backtrace: 2021-11-23T20:24:54.645 INFO:teuthology.orchestra.run.smithi124.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2021-11-23T20:24:54.653 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:24:54.701+0000 7f3db36e4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 18048) UID: 0 Ignoring malparsed backtrace: 2021-11-23T20:24:56.804 INFO:teuthology.orchestra.run.smithi124.stdout: "size": 20, , 2021-11-23T20:24:56.805 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 600, , 2021-11-23T20:24:56.805 INFO:teuthology.orchestra.run.smithi124.stdout: "ops": [ Ignoring malparsed backtrace: 2021-11-23T20:24:56.827 INFO:teuthology.orchestra.run.smithi124.stdout: }, , 2021-11-23T20:24:56.827 INFO:teuthology.orchestra.run.smithi124.stdout: { , 2021-11-23T20:24:56.827 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "throttled", Ignoring malparsed backtrace: 2021-11-23T20:24:56.853 INFO:teuthology.orchestra.run.smithi124.stdout: }, , 2021-11-23T20:24:56.853 INFO:teuthology.orchestra.run.smithi124.stdout: { , 2021-11-23T20:24:56.854 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "queued_for_pg", Ignoring malparsed backtrace: 2021-11-23T20:24:56.878 INFO:teuthology.orchestra.run.smithi124.stdout: }, , 2021-11-23T20:24:56.879 INFO:teuthology.orchestra.run.smithi124.stdout: { , 2021-11-23T20:24:56.879 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "op_commit", Ignoring malparsed backtrace: 2021-11-23T20:24:56.907 INFO:teuthology.orchestra.run.smithi124.stdout: } , 2021-11-23T20:24:56.908 INFO:teuthology.orchestra.run.smithi124.stdout: ] , 2021-11-23T20:24:56.908 INFO:teuthology.orchestra.run.smithi124.stdout: } Ignoring malparsed backtrace: 2021-11-23T20:24:56.930 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 0 , 2021-11-23T20:24:56.930 INFO:teuthology.orchestra.run.smithi124.stdout: }, , 2021-11-23T20:24:56.930 INFO:teuthology.orchestra.run.smithi124.stdout: { Ignoring malparsed backtrace: 2021-11-23T20:24:56.956 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 1.1491000000000001e-05 , 2021-11-23T20:24:56.957 INFO:teuthology.orchestra.run.smithi124.stdout: }, , 2021-11-23T20:24:56.957 INFO:teuthology.orchestra.run.smithi124.stdout: { Ignoring malparsed backtrace: 2021-11-23T20:24:56.983 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 0.033520279 , 2021-11-23T20:24:56.983 INFO:teuthology.orchestra.run.smithi124.stdout: }, , 2021-11-23T20:24:56.984 INFO:teuthology.orchestra.run.smithi124.stdout: { Ignoring malparsed backtrace: 2021-11-23T20:24:57.010 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 4.0519000000000003e-05 , 2021-11-23T20:24:57.010 INFO:teuthology.orchestra.run.smithi124.stdout: } , 2021-11-23T20:24:57.011 INFO:teuthology.orchestra.run.smithi124.stdout: ] Ignoring malparsed backtrace: 2021-11-23T20:24:57.037 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:51.067092+0000", , 2021-11-23T20:24:57.037 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 5.6760000000000004e-06 , 2021-11-23T20:24:57.037 INFO:teuthology.orchestra.run.smithi124.stdout: }, Ignoring malparsed backtrace: 2021-11-23T20:24:57.061 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:51.113800+0000", , 2021-11-23T20:24:57.062 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 3.8843000000000001e-05 , 2021-11-23T20:24:57.062 INFO:teuthology.orchestra.run.smithi124.stdout: } Ignoring malparsed backtrace: 2021-11-23T20:24:57.090 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "dispatched", , 2021-11-23T20:24:57.090 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:51.072822+0000", , 2021-11-23T20:24:57.090 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 1.0661e-05 Ignoring malparsed backtrace: 2021-11-23T20:24:57.113 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "waiting for subops from 6", , 2021-11-23T20:24:57.113 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:51.101185+0000", , 2021-11-23T20:24:57.113 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 0.000127432 Ignoring malparsed backtrace: 2021-11-23T20:24:57.139 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "done", , 2021-11-23T20:24:57.139 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:51.118656+0000", , 2021-11-23T20:24:57.139 INFO:teuthology.orchestra.run.smithi124.stdout: "duration": 3.2376000000000003e-05 Ignoring malparsed backtrace: 2021-11-23T20:24:57.164 INFO:teuthology.orchestra.run.smithi124.stdout: { , 2021-11-23T20:24:57.164 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "initiated", , 2021-11-23T20:24:57.165 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:56.057694+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.195 INFO:teuthology.orchestra.run.smithi124.stdout: { , 2021-11-23T20:24:57.196 INFO:teuthology.orchestra.run.smithi124.stdout: "event": "dispatched", , 2021-11-23T20:24:57.196 INFO:teuthology.orchestra.run.smithi124.stdout: "time": "2021-11-23T20:22:56.102690+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.528 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", , 2021-11-23T20:24:57.528 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:04.880161+0000", , 2021-11-23T20:24:57.529 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8630999999999999e-05 Ignoring malparsed backtrace: 2021-11-23T20:24:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", , 2021-11-23T20:24:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:04.880516+0000", , 2021-11-23T20:24:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011037 Ignoring malparsed backtrace: 2021-11-23T20:24:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", , 2021-11-23T20:24:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:05.026275+0000", , 2021-11-23T20:24:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5777000000000002e-05 Ignoring malparsed backtrace: 2021-11-23T20:24:57.598 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", , 2021-11-23T20:24:57.598 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:06.004313+0000", , 2021-11-23T20:24:57.598 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.975670963 Ignoring malparsed backtrace: 2021-11-23T20:24:57.620 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5", , 2021-11-23T20:24:57.620 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:05.136767+0000", , 2021-11-23T20:24:57.620 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000127255 Ignoring malparsed backtrace: 2021-11-23T20:24:57.643 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", , 2021-11-23T20:24:57.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:05.158003+0000", , 2021-11-23T20:24:57.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0052390450000000003 Ignoring malparsed backtrace: 2021-11-23T20:24:57.665 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", , 2021-11-23T20:24:57.665 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:06.035560+0000", , 2021-11-23T20:24:57.665 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00087594500000000004 Ignoring malparsed backtrace: 2021-11-23T20:24:57.687 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 172.35436948700001, , 2021-11-23T20:24:57.687 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.87921918099999996, , 2021-11-23T20:24:57.688 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { Ignoring malparsed backtrace: 2021-11-23T20:24:57.709 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:57.709 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", , 2021-11-23T20:24:57.710 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:05.169108+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.731 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:57.732 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", , 2021-11-23T20:24:57.732 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:05.172516+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.753 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:57.753 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", , 2021-11-23T20:24:57.754 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:05.187167+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.775 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:57.776 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", , 2021-11-23T20:24:57.776 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:05.201390+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", , 2021-11-23T20:24:57.798 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:06.018839+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.820 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:57.820 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 4", , 2021-11-23T20:24:57.820 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:06.020546+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.842 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:57.842 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", , 2021-11-23T20:24:57.842 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:06.048032+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.864 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:57.864 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", , 2021-11-23T20:24:57.864 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:22:06.054857+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:57.886 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2021-11-23T20:22:05.370019+0000", , 2021-11-23T20:24:57.886 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 172.143313482, , 2021-11-23T20:24:57.886 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.68900967899999999, Ignoring malparsed backtrace: 2021-11-23T20:24:57.907 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ , 2021-11-23T20:24:57.907 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:57.907 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", Ignoring malparsed backtrace: 2021-11-23T20:24:58.021 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 , 2021-11-23T20:24:58.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2021-11-23T20:24:58.021 INFO:teuthology.orchestra.run.smithi103.stdout: { Ignoring malparsed backtrace: 2021-11-23T20:24:58.301 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:24:28.069799+0000", , 2021-11-23T20:24:58.301 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 , 2021-11-23T20:24:58.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, Ignoring malparsed backtrace: 2021-11-23T20:24:58.327 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.352356909, , 2021-11-23T20:24:58.328 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { , 2021-11-23T20:24:58.328 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", Ignoring malparsed backtrace: 2021-11-23T20:24:58.349 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", , 2021-11-23T20:24:58.349 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:24:33.477756+0000", , 2021-11-23T20:24:58.350 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.012161407000000001 Ignoring malparsed backtrace: 2021-11-23T20:24:58.379 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:58.379 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", , 2021-11-23T20:24:58.379 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:24:34.412239+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:58.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2021-11-23T20:24:58.401 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:58.402 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", Ignoring malparsed backtrace: 2021-11-23T20:24:58.423 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 , 2021-11-23T20:24:58.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2021-11-23T20:24:58.423 INFO:teuthology.orchestra.run.smithi103.stdout: { Ignoring malparsed backtrace: 2021-11-23T20:24:58.445 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { , 2021-11-23T20:24:58.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", , 2021-11-23T20:24:58.445 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ Ignoring malparsed backtrace: 2021-11-23T20:24:58.466 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:24:47.771520+0000", , 2021-11-23T20:24:58.466 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0095869460000000007 , 2021-11-23T20:24:58.466 INFO:teuthology.orchestra.run.smithi103.stdout: } Ignoring malparsed backtrace: 2021-11-23T20:24:58.488 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", , 2021-11-23T20:24:58.488 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:24:48.450623+0000", , 2021-11-23T20:24:58.488 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.34074589799999999 Ignoring malparsed backtrace: 2021-11-23T20:24:58.509 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:58.510 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", , 2021-11-23T20:24:58.510 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2021-11-23T20:24:51.811863+0000", Ignoring malparsed backtrace: 2021-11-23T20:24:58.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2021-11-23T20:24:58.531 INFO:teuthology.orchestra.run.smithi103.stdout: { , 2021-11-23T20:24:58.531 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", Ignoring malparsed backtrace: 2021-11-23T20:24:58.805 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], , 2021-11-23T20:24:58.805 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 0 , 2021-11-23T20:24:58.805 INFO:teuthology.orchestra.run.smithi103.stdout:} Ignoring malparsed backtrace: 2021-11-23T20:24:59.986 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:24:59.987 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:25:00.015 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:25:00.422 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:00.423 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:00.945 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: 2021-11-23T20:25:05.460 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:25:05.489 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:25:05.997 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:06.462 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:06.464 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:25:06.925 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:06.926 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:25:12.989 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:25:13.495 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:25:13.994 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:13.995 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:25:14.002 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:25:14.450 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:14.451 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:25:14.509 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:14.994 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:15.431 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:25:21.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:25:22.007 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:25:22.445 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:25:22.513 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:22.986 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:22.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:23.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:23.443 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:25.022 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:25:30.076 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:25:30.576 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:30.577 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:25:30.584 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:25:31.012 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:25:31.048 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 226 16 86688 86672 23.9664 0 - 0.0133532 , 2021-11-23T20:25:31.090 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:31.559 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:32.010 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:32.011 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:25:38.077 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:38.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:25:38.085 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:25:38.528 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:25:38.592 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:25:39.049 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 234 16 86688 86672 23.147 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:39.072 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:39.533 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:39.534 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:25:40.049 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 235 16 86688 86672 23.0485 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: 2021-11-23T20:25:45.059 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:25:45.088 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:45.594 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:45.595 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:25:46.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:46.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:25:46.050 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 241 16 86688 86672 22.4747 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:47.044 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:25:52.702 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:25:53.208 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:25:53.703 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:53.704 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:25:53.716 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:25:54.154 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:25:54.155 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:25:54.223 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:54.705 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:25:55.162 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:26:01.734 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:26:01.734 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:26:02.183 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:26:02.245 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:02.733 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:02.734 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:03.227 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:03.228 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:26:03.705 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:03.706 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:26:03.771 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:04.700 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:26:11.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:26:11.300 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:11.791 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:11.791 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:12.233 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:12.234 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:26:20.894 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:26:20.929 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:26:21.352 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:21.439 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:21.918 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:21.919 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:22.365 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:22.366 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:26:27.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:26:27.437 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:26:27.945 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:28.437 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:28.440 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:26:28.885 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:28.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:26:28.974 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:26:34.976 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:26:35.484 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:26:35.973 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:35.974 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:26:35.992 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:26:36.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:36.435 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:26:36.500 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:36.974 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:37.431 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:45.063 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:45.064 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:26:45.523 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:45.526 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:26:45.618 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:26:46.060 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 301 16 86688 86672 17.9945 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:26:51.607 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:26:52.114 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:26:52.608 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:52.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:26:52.621 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:26:53.065 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:26:53.066 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:26:53.129 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:53.601 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: 2021-11-23T20:26:54.061 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 309 16 86688 86672 17.5286 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:26:54.067 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:26:55.061 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 310 16 86688 86672 17.4721 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:27:00.407 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:27:00.436 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:00.938 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:00.939 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:27:06.926 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:07.418 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:07.419 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:27:07.888 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:07.889 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:27:07.940 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:08.886 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:27:14.499 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:27:15.497 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:15.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:27:15.513 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:27:15.954 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:27:16.019 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:16.494 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:16.960 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:16.961 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:27:23.512 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:27:23.530 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:27:24.038 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:24.503 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:24.504 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:27:24.960 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:24.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:27:32.010 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:27:32.063 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:32.555 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:33.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:33.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:27:40.600 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:41.079 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:41.080 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:27:41.543 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:41.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:27:41.615 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:27:42.069 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 357 16 86688 86672 15.1718 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:27:48.696 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:27:48.767 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:49.239 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:49.696 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:49.697 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:27:56.254 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:27:56.296 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:56.789 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:56.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:27:57.247 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:27:57.248 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:28:04.307 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:05.272 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:28:12.433 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:28:12.472 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:28:12.982 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:13.431 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:28:13.432 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:28:20.492 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:28:20.988 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:28:20.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:28:20.999 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:28:21.435 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:28:21.435 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:28:21.506 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:21.992 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:22.436 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:28:27.990 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:28:27.992 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:28:28.434 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:28:28.499 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:29.166 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:29.635 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:28:29.635 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:28:35.668 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:28:35.710 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:28:36.128 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:28:36.218 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:36.709 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:28:36.710 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:37.160 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:28:37.161 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:28:44.237 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:28:44.743 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:45.201 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:28:52.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:28:52.389 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:52.855 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:28:52.856 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:28:53.315 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:28:53.316 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:28:59.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:28:59.391 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:28:59.898 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:00.375 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:00.377 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:29:00.830 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:00.831 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:29:00.913 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: 2021-11-23T20:29:05.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:29:05.370 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:29:05.879 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:06.349 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:06.350 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:29:13.383 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:29:14.354 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:14.355 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:29:14.396 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:15.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:15.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:29:21.538 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:21.539 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:29:21.564 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:29:21.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:29:22.072 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:22.533 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:22.991 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:22.992 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:29:28.534 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:29:28.562 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:29.059 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:29.060 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:29.517 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:29.519 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:31.021 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:29:36.082 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:29:36.590 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:29:37.048 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:37.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:29:37.087 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 472 16 86688 86672 11.4752 0 - 0.0133532 Ignoring malparsed backtrace: 2021-11-23T20:29:37.505 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:37.507 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:38.032 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:29:45.089 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 480 16 86688 86672 11.2839 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:45.139 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:45.588 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:45.589 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:29:46.088 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 481 16 86688 86672 11.2605 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:29:52.123 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:29:52.156 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:29:52.663 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:29:53.136 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:29:53.138 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:29:54.090 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 489 16 86688 86672 11.0763 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:29:59.669 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:30:00.177 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:30:00.655 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:00.656 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:30:00.684 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:30:01.092 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:01.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:01.629 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:30:07.629 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:30:07.681 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:30:08.092 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 503 16 86688 86672 10.768 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:08.167 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:08.623 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:08.625 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:30:15.828 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:30:16.334 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:30:16.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:16.820 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:30:16.840 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:30:17.267 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:17.269 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:30:17.346 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:17.808 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:30:23.345 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:30:24.350 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:24.351 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:30:24.360 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:30:24.785 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:30:24.868 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:30:31.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:30:31.374 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:30:31.882 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:32.373 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:32.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:30:32.837 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:32.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:30:32.898 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:30:33.314 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:33.316 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:30:33.859 DEBUG:teuthology.orchestra.run:got remote process result: 22 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:34.328 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:30:42.071 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:30:42.578 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:30:43.055 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:43.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:30:43.085 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:30:43.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:43.554 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:30:43.592 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:44.081 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:44.541 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:30:50.552 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:30:50.596 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:30:51.099 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 546 16 86688 86672 9.91993 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:51.104 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:51.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:51.554 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:30:58.191 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:30:58.214 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:30:58.722 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:30:59.189 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:30:59.191 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:31:05.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:31:05.719 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:06.212 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:06.672 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:31:06.673 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:31:11.712 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:31:12.219 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:31:12.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:31:12.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:31:12.729 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:31:13.141 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:31:13.142 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:31:13.680 DEBUG:teuthology.orchestra.run:got remote process result: 22 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:31:19.234 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:31:20.205 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:31:20.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:31:20.252 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:31:20.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:20.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:21.201 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:31:27.243 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:31:27.262 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:31:28.312 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:31:28.821 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:31:29.276 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:31:29.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:31:29.328 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:31:29.741 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:29.824 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:30.275 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:31:36.800 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:31:36.837 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:31:37.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:31:37.346 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:37.804 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:31:37.805 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:31:44.330 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:31:44.357 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:44.862 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:31:44.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:45.790 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:31:51.371 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:31:51.863 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:31:51.866 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:31:51.879 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:31:52.322 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:31:52.323 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:31:52.390 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:52.854 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:31:53.317 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:32:00.947 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:32:01.456 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:32:01.935 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:01.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:32:01.964 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:32:02.395 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:02.396 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:32:02.476 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:02.932 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:03.393 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:32:08.444 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:08.445 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:32:08.470 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:32:08.906 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:32:08.979 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:09.436 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:09.900 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:09.901 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:32:16.482 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:32:16.496 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:32:17.004 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:17.480 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:17.481 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:32:17.934 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:17.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:32:18.020 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:32:19.164 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:32:24.756 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:32:25.183 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:25.185 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:32:25.263 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:25.725 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:32:32.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:32:32.769 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:32:33.194 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:32:33.277 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:32:33.741 DEBUG:teuthology.orchestra.run:got remote process result: 22 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:33.743 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:34.195 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:34.196 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:32:40.783 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:32:41.290 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:32:41.741 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:41.742 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:32:41.796 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:42.731 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:32:47.277 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:32:48.252 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:48.253 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:32:48.291 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:32:48.710 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:48.793 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:49.268 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:50.425 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:50.426 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:32:56.483 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:32:56.493 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:32:57.000 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:57.479 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:57.481 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:32:57.940 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:32:57.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:32:58.013 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:32:58.930 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:33:04.002 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:33:04.507 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:33:05.002 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:05.003 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:33:05.013 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:05.445 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:33:09.978 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:33:10.486 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:33:10.951 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:10.952 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:33:10.994 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:33:11.405 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:11.406 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:11.948 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:33:18.498 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:18.499 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:33:18.511 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:33:18.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:33:19.018 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:19.495 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:19.954 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:19.955 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:33:25.128 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:25.129 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:33:25.130 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:33:25.590 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:33:25.636 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:33:26.122 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 701 16 86688 86672 7.72647 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:26.126 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:26.591 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:26.592 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:33:27.122 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 702 16 86688 86672 7.71547 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:33:33.148 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:33:33.149 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:33:33.657 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:34.131 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:34.132 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:33:34.597 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:34.598 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:33:34.670 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:35.124 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 710 16 86688 86672 7.62853 0 - 0.0133532 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:33:40.657 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:41.159 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:41.160 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:33:41.623 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:41.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:33:41.669 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:33:42.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:33:42.124 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 717 16 86688 86672 7.55405 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:42.627 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:33:47.156 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:33:48.133 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:48.134 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:33:48.170 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:33:48.597 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:33:48.678 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:33:49.125 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 724 16 86688 86672 7.48102 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:49.135 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:49.599 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:49.600 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:33:50.126 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 725 16 86688 86672 7.4707 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:33:54.358 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:33:55.321 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:33:55.322 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:33:55.374 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:55.871 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:33:56.321 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:34:02.834 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:34:02.880 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:34:03.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:34:03.387 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:03.850 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:03.851 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:04.301 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:04.302 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:34:09.343 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:34:09.377 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:34:09.885 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:10.354 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:10.355 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:34:10.809 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:10.811 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:34:17.394 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:34:17.889 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:17.891 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:34:17.902 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:34:18.332 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:18.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:34:18.410 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:18.881 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:19.328 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:34:25.131 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) , 2021-11-23T20:34:25.132 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 760 16 86688 86672 7.12665 0 - 0.0133532 , 2021-11-23T20:34:25.151 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:34:25.589 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:34:25.660 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:34:26.131 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 761 16 86688 86672 7.11728 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:26.138 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:26.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:26.586 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:34:32.651 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:34:32.658 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:34:33.132 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 768 16 86688 86672 7.05241 0 - 0.0133532 , 2021-11-23T20:34:33.165 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:33.641 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:33.642 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:34:34.093 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:34.094 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:34:34.132 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 769 16 86688 86672 7.04324 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:34:40.140 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:34:40.159 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:34:40.666 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:34:41.128 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:41.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:34:41.132 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 776 16 86688 86672 6.97971 0 - 0.0133532 Ignoring malparsed backtrace: 2021-11-23T20:34:41.580 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:41.582 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:42.122 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:34:47.159 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:34:48.149 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:48.150 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:34:48.172 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:34:48.601 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:34:48.681 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:34:49.134 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 784 16 86688 86672 6.90848 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:49.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:49.605 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:49.606 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:34:50.135 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 785 16 86688 86672 6.89968 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:34:57.273 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:34:58.254 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:34:58.255 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:34:58.287 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:34:58.701 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:58.792 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:34:59.248 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:35:07.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:35:07.832 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:35:08.285 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:35:08.342 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:08.827 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:08.827 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:35:14.812 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:35:14.835 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:35:15.264 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:35:15.341 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:15.800 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:15.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:35:22.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:35:22.348 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:35:22.856 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:23.338 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:23.339 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:23.913 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:35:29.495 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:35:30.474 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:30.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:35:30.511 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:35:30.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:31.018 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:31.486 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:35:38.033 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:35:38.040 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:35:38.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:35:38.546 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:39.021 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:39.022 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:39.481 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:39.482 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:40.989 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:35:46.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:46.554 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:35:46.565 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:35:47.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:35:47.072 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:47.541 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:47.997 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:47.998 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:35:55.146 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:35:55.146 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:35:55.653 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:56.150 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:56.151 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:35:56.601 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:35:56.604 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:35:56.668 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:35:57.144 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 852 16 86688 86672 6.3571 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:35:57.615 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:36:03.166 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:36:03.650 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:03.651 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:36:03.672 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:36:04.101 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:36:04.145 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 859 16 86688 86672 6.30529 0 - 0.0133532 , 2021-11-23T20:36:04.179 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:04.638 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:36:09.158 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:36:10.154 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:10.155 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:36:10.173 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:36:10.603 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:36:10.679 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:11.146 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:11.147 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:11.607 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:11.608 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:36:17.652 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:36:17.680 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:36:18.147 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 873 16 86688 86672 6.20418 0 - 0.0133532 , 2021-11-23T20:36:18.189 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:18.647 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:18.647 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:36:19.106 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:19.107 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:36:19.147 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 874 16 86688 86672 6.19708 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:36:25.149 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2021-11-23T20:36:25.147740+0000 min lat: 0.00170168 max lat: 3.50034 avg lat: 0.0133532 , 2021-11-23T20:36:25.149 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) , 2021-11-23T20:36:25.149 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 880 16 86688 86672 6.15482 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:36:31.707 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:36:32.214 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:36:32.701 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:32.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:36:32.721 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:36:33.154 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:33.155 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:36:33.229 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:33.695 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:36:40.217 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:40.218 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:36:40.232 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:36:40.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:36:40.739 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:36:41.150 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 896 16 86688 86672 6.04492 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:41.213 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:41.717 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:41.718 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:36:47.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:36:47.244 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:36:47.749 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:48.244 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:48.245 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:36:48.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:48.691 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:36:48.762 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:49.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:36:56.930 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:36:56.953 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:36:57.387 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:36:57.462 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:57.932 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:57.933 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:36:58.391 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:36:58.392 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:37:05.436 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:05.437 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:37:05.479 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:37:05.887 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:05.980 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:06.426 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:37:12.468 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:37:12.484 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:37:12.991 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:13.450 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:13.911 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:13.912 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:37:19.982 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:37:19.990 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:37:20.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:37:20.496 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:20.957 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:20.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:21.417 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:21.418 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:37:26.995 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:37:27.501 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:37:28.009 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:37:28.454 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:28.455 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:37:28.515 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:29.003 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:29.461 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:37:35.516 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:35.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:37:35.520 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:37:35.992 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:37:36.027 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:37:36.442 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:36.530 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:36.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:36.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:37:43.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:37:43.548 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:37:44.055 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:44.532 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:44.534 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:44.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:37:45.070 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:37:50.548 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:37:51.522 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:51.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:37:51.563 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:52.041 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:37:52.508 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:37:58.197 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:37:59.192 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:37:59.194 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:37:59.213 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:37:59.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:37:59.722 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:38:00.162 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 975 16 86688 86672 5.55512 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:00.200 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:00.672 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:00.674 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:38:01.162 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 976 16 86688 86672 5.54943 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:38:06.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:38:06.226 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:06.722 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:06.723 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:07.184 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:07.185 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:08.163 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:38:13.733 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:14.218 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:14.220 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:38:14.672 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:14.673 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:38:14.748 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:38:15.165 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 990 16 86688 86672 5.47095 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:15.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:38:22.246 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:22.247 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:38:22.257 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:38:22.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:38:22.765 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:23.232 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:23.734 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:23.735 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:38:30.329 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:38:30.804 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:30.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:38:30.833 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:38:31.254 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:31.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:38:31.341 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:31.800 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:38:38.339 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:38:38.780 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:38:38.847 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:38:39.354 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:39.814 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:39.815 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:40.269 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:40.270 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: 2021-11-23T20:38:45.331 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:38:45.335 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:38:45.842 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:46.310 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:46.311 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:38:46.764 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:46.765 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:47.802 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:38:53.342 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:38:54.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:38:54.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:38:54.355 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:54.842 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:38:55.282 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:39:01.550 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:01.551 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:39:01.558 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:39:01.993 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:39:02.066 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:02.526 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:39:08.027 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:39:08.052 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:39:08.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:39:08.559 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:09.005 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:09.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:39:14.544 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:39:14.546 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:39:15.054 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:15.524 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:15.974 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:15.975 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:39:22.558 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:39:22.559 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:39:23.066 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:23.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:23.554 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:39:23.998 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:24.000 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:39:24.083 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:25.017 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:39:31.131 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:39:31.132 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:39:31.590 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:39:31.639 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:32.132 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:32.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:32.633 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:32.634 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:39:33.080 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:33.082 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:39:33.163 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:34.093 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:39:41.187 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:41.189 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:39:41.190 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:39:41.643 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:39:41.697 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:42.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:42.172 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:42.633 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:42.635 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:39:43.178 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1078 16 86688 86672 5.02434 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:39:49.213 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:39:49.713 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:49.714 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:39:49.721 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:39:50.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:39:50.179 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1085 16 86688 86672 4.99192 0 - 0.0133532 , 2021-11-23T20:39:50.228 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:50.723 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:51.179 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1086 16 86688 86672 4.98732 0 - 0.0133532 , 2021-11-23T20:39:51.180 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:39:57.733 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:39:57.740 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:39:58.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:39:58.246 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:39:58.707 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:58.709 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:39:59.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:39:59.172 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:39:59.180 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1094 16 86688 86672 4.95085 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:40:06.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:40:06.850 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:40:07.277 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:40:07.360 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:07.813 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:07.814 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:40:14.869 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:15.371 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:15.372 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:40:15.826 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:15.827 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:40:15.883 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:16.810 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:40:23.861 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:40:23.899 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:40:24.309 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:40:24.406 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:24.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:24.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:40:30.907 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:31.415 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:40:32.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:32.039 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:40:32.055 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:40:32.472 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:32.473 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:33.012 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:40:39.564 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:39.565 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:40:39.572 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:40:40.011 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:40:40.081 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:40.550 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:41.009 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:41.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:40:46.571 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:40:46.601 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:40:47.111 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:47.586 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:47.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:40:48.036 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:48.037 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:40:54.641 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:40:55.638 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:55.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:40:55.658 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:40:56.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:40:56.166 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:56.640 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:40:57.088 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:40:57.089 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:41:04.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:41:04.320 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:41:04.828 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:05.765 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:41:10.814 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:41:11.323 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:41:11.807 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:11.808 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:41:11.830 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:41:12.264 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:12.266 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:41:12.338 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:12.806 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:13.272 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:41:17.828 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:18.320 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:18.321 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:41:18.773 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:18.774 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:41:18.845 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:19.782 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:41:25.832 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:41:25.858 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:41:26.296 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:41:26.370 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:26.834 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:26.835 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:27.294 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:27.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:29.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:41:34.556 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:41:35.575 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:41:36.023 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:41:36.083 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:36.580 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:37.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:37.021 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:41:42.594 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:41:43.053 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:41:43.102 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:43.600 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:44.042 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:44.043 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:41:49.575 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:41:49.600 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:50.105 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:50.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:50.552 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:50.553 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:41:57.616 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:41:58.121 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:58.123 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:41:58.124 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:41:58.582 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:41:58.583 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:41:58.631 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:41:59.740 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:42:04.797 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:42:05.238 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:05.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:42:05.303 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:05.789 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:06.236 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:42:12.746 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:42:12.807 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:13.285 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:13.735 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:13.736 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:42:20.302 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:42:20.317 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:42:20.826 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:21.300 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:21.301 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:42:21.751 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:21.752 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:42:27.475 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:42:27.977 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:27.978 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:42:27.982 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:42:28.419 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:28.420 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:42:28.490 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:28.972 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:29.418 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:42:35.494 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:35.495 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:42:35.496 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:42:35.942 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:42:36.002 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:36.486 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:36.942 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:36.943 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:42:43.487 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:42:43.514 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:42:44.023 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:44.500 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:44.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:42:44.939 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:44.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:42:50.511 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:42:50.997 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:50.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:42:51.018 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:42:51.443 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:51.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:42:51.524 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:51.997 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:52.453 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:42:57.575 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:42:58.566 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:42:58.567 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:42:58.589 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:42:59.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:42:59.099 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:42:59.571 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:00.030 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:00.031 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: 2021-11-23T20:43:05.549 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:43:05.592 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:06.076 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:06.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:06.538 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:06.540 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:08.052 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:43:13.612 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:43:14.603 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:14.605 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:43:14.625 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:43:15.048 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:43:15.133 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:43:15.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:43:20.246 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:43:20.253 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:43:20.760 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:21.244 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:21.245 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:43:21.700 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:21.701 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:43:21.775 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:43:22.210 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1297 16 86688 86672 4.17596 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:43:27.265 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:43:28.209 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:43:28.211 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1303 16 86688 86672 4.15673 0 - 0.0133532 , 2021-11-23T20:43:28.279 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:28.753 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:43:29.193 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:29.194 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:43:35.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:43:35.783 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:43:36.227 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:43:36.290 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:36.772 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:36.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:43:37.211 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:37.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:43:37.213 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1312 16 86688 86672 4.12822 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:43:42.886 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:43:42.887 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:43:43.343 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:43:43.393 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:43.880 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:43.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:44.339 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:44.340 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:43:49.887 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:43:50.392 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:43:50.859 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:50.860 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:43:50.899 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:51.836 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:43:58.856 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:43:58.910 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:59.392 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:43:59.851 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:43:59.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:44:05.908 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:44:06.415 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:44:07.086 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:07.087 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:44:07.095 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:44:07.543 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:07.544 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:44:07.603 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:08.087 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:08.541 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:44:15.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:44:15.106 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:44:15.544 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:44:15.614 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:16.082 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:16.083 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:16.549 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:16.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:44:22.622 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:44:22.623 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:44:23.130 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:23.626 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:23.627 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:44:24.081 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:24.082 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:44:24.148 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:25.073 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:44:29.636 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:44:30.608 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:30.610 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:44:30.651 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:44:31.068 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:31.151 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:31.599 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:44:37.770 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:37.771 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:44:37.780 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:44:38.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:44:38.221 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1373 16 86688 86672 3.94481 0 - 0.0133532 , 2021-11-23T20:44:38.289 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:38.768 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: 2021-11-23T20:44:39.221 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1374 16 86688 86672 3.94194 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:39.228 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:44:45.223 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1380 16 86688 86672 3.9248 0 - 0.0133532 , 2021-11-23T20:44:45.301 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:45.760 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:45.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:44:46.222 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1381 16 86688 86672 3.92196 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:44:51.801 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:44:51.801 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:44:52.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:44:52.309 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:52.783 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:52.784 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:44:53.243 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:44:53.244 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:44:59.821 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:00.316 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:00.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:45:00.775 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:00.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:45:00.838 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:01.758 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:45:08.358 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:45:08.405 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:45:08.913 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:09.366 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:09.367 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:45:09.832 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:09.834 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:45:16.430 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:45:17.399 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:17.400 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:45:17.444 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:45:17.852 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:17.940 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:18.391 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:45:23.431 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:45:24.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:24.442 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:45:24.446 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:45:24.892 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:45:24.954 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:45:31.468 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:45:31.496 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:45:32.005 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:32.493 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:32.494 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:45:32.951 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:32.952 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:45:33.020 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:45:39.532 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:45:40.510 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:40.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:45:40.546 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:45:40.969 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:41.053 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:41.499 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:45:47.507 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:45:47.560 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:48.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:48.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:48.514 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:48.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:50.110 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:45:55.694 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:45:56.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:56.691 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:45:56.712 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:45:57.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:45:57.220 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:57.703 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:45:58.165 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:45:58.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:46:04.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:46:04.834 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:05.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:05.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:46:05.750 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:05.751 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:46:13.864 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:46:13.890 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:46:14.344 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:46:14.403 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:14.892 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:14.894 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:15.342 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:15.343 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:46:22.453 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:46:22.955 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:22.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:46:22.961 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:46:23.414 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:23.415 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:46:23.469 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:23.963 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:24.411 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:46:29.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:46:29.972 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:46:30.405 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:46:30.481 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:30.942 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:30.944 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:46:31.396 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:31.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:46:38.605 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:46:39.625 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:39.626 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:46:39.627 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:46:40.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:46:40.133 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:40.630 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:41.073 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:41.074 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:46:46.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:46:46.641 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:46:47.149 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:47.638 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:47.639 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:46:48.102 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:48.103 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:46:48.165 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:46:48.599 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:48.600 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:46:48.673 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:49.134 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:49.596 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:46:56.159 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:46:56.186 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:46:56.618 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:46:56.694 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:46:57.148 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:46:57.149 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:05.715 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:47:12.844 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:47:12.852 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:47:13.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:47:13.361 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:13.835 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:47:13.837 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:14.298 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:47:14.299 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:47:20.365 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:47:20.367 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:47:20.875 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:21.353 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:47:21.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:21.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:47:25.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:47:25.851 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:47:26.300 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:47:26.356 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:47:26.865 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:47:27.372 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:27.837 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:47:27.879 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:28.830 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:47:34.384 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:47:35.349 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:47:35.350 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:47:35.405 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:35.894 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:36.353 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:47:41.475 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:47:43.478 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:47:43.507 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:47:44.015 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:47:44.523 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:47:45.002 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:47:45.004 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:47:45.029 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:45.456 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:47:52.037 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:47:52.040 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:47:53.069 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:47:53.577 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:47:54.044 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:47:54.045 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:47:54.085 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:47:54.493 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:47:54.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:47:55.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:48:00.584 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:48:01.088 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:01.089 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:48:01.090 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:48:01.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:01.554 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:48:01.597 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:48:02.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:02.103 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:02.550 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:48:08.198 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:48:08.210 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:48:08.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:48:08.717 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:09.212 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:09.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:09.674 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:09.675 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:48:16.740 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:48:17.243 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:17.245 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:48:17.250 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:48:17.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:17.738 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:48:17.757 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:48:18.202 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:48:18.252 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1593 16 86688 86672 3.40001 0 - 0.0133532 , 2021-11-23T20:48:18.265 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:18.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:19.189 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:19.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:48:25.254 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1600 16 86688 86672 3.38514 0 - 0.0133532 , 2021-11-23T20:48:25.266 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:25.749 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:25.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:48:26.215 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:26.216 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:48:26.254 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1601 16 86688 86672 3.38302 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:27.225 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:27.725 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:48:33.278 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:48:34.266 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:34.267 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:48:34.291 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:48:34.715 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:48:34.799 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:48:35.255 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1610 16 86688 86672 3.36411 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:35.261 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:48:41.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:48:41.917 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:42.416 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:42.865 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:42.866 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:48:48.935 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:49.427 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:49.429 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:48:49.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:49.896 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:48:49.949 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:48:50.357 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:50.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:50.893 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:48:56.446 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:48:57.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:57.434 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:48:57.463 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:48:57.891 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:48:57.972 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:58.446 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:48:58.901 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:48:58.902 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:49:04.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:49:04.982 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:05.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:05.434 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:49:12.487 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:49:13.487 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:13.488 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:49:13.503 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:49:13.943 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:49:14.012 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:14.493 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:14.934 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:14.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:49:21.486 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:49:21.524 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:49:22.033 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:22.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:22.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:49:27.457 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:49:27.510 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:27.993 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:27.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:28.447 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:28.449 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:49:34.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:49:34.514 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:49:35.021 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:35.499 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:35.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:49:35.969 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:35.970 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:49:36.036 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:36.969 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:49:43.645 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:49:44.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:49:44.151 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:44.628 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:45.088 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:45.089 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:49:50.604 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:49:50.638 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:51.133 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:51.134 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:49:51.590 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:51.591 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:49:58.659 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:49:59.658 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:49:59.660 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:49:59.674 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:50:00.103 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:50:00.183 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:00.659 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:01.116 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:01.117 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:50:06.631 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:50:06.673 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:50:07.181 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:07.633 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:07.634 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:50:14.311 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:50:14.315 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:50:14.822 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:15.304 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:15.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:50:15.750 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:15.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:50:15.835 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:50:16.270 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1711 16 86688 86672 3.16553 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:16.753 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:50:22.336 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:50:23.317 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:23.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:50:23.352 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:50:23.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:23.850 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: 2021-11-23T20:50:24.271 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1719 16 86688 86672 3.1508 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:24.297 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:50:30.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:50:30.361 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:30.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:30.853 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:31.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:31.291 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:50:37.402 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:50:37.417 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:50:37.925 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:38.398 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:38.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:50:38.860 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:38.862 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:50:38.940 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:39.867 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:50:44.949 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:50:45.391 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:45.392 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:50:45.456 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:45.936 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:46.395 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:50:53.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:50:53.497 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:53.992 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:53.993 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:50:54.448 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:50:54.449 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:51:04.177 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:04.685 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:51:05.140 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:05.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:51:05.192 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:05.602 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:51:10.699 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:51:12.190 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:51:12.226 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:12.717 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:13.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:13.170 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:51:20.970 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:21.463 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:21.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:51:21.911 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:21.913 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:22.921 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:51:27.493 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:51:27.965 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:27.966 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:51:28.003 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:51:28.413 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:28.414 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:28.960 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:51:36.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:51:36.068 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:51:36.513 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:51:36.574 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:37.064 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:37.065 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:37.522 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:37.523 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:39.581 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:40.038 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:40.039 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: 2021-11-23T20:51:45.606 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:51:45.607 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:51:46.115 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:51:46.622 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:51:47.081 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:47.082 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:51:47.130 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:48.090 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:51:54.662 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:54.663 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:51:54.666 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:51:55.097 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:51:55.172 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:55.649 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:51:56.111 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:51:56.113 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:52:04.443 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:05.411 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:52:11.449 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:11.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:52:11.454 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:52:11.898 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:52:11.961 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:12.424 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:12.881 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:12.882 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:52:19.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:52:19.473 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:52:19.981 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:20.452 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:20.453 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:52:20.913 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:20.914 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:52:20.995 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:21.960 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:52:26.474 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:52:27.476 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:27.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:52:27.490 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:52:27.926 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:52:27.996 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:28.513 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:28.970 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:28.972 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:52:38.094 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:52:38.098 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:52:38.557 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:52:38.605 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:39.099 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:39.100 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:39.563 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:39.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: 2021-11-23T20:52:45.619 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:52:46.122 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:46.123 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:52:46.128 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:52:46.578 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:46.580 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:52:46.634 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:47.120 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:47.572 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:52:54.125 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T20:52:54.138 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:52:54.566 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:52:54.647 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:52:55.112 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:55.113 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:52:55.562 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:52:55.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:53:01.759 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T20:53:01.765 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:53:02.272 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:02.757 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:02.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:53:03.220 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:03.221 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:53:03.288 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:04.243 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: 2021-11-23T20:53:04.711 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:53:04.810 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:05.279 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:05.280 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:53:11.811 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:53:12.319 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:53:12.778 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:12.780 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:53:12.828 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:53:13.239 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:53:13.296 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1888 16 86688 86672 2.86876 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:13.773 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:53:20.321 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:20.322 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:53:20.344 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:53:20.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:53:20.851 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:53:21.298 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1896 16 86688 86672 2.85665 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:21.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:21.782 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:21.783 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:53:22.298 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1897 16 86688 86672 2.85515 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:53:26.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:53:26.835 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:53:27.298 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1902 16 86688 86672 2.84764 0 - 0.0133532 , 2021-11-23T20:53:27.342 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:53:27.780 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:27.781 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:53:27.850 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:53:28.299 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1903 16 86688 86672 2.84615 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:53:34.968 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:53:35.478 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:53:36.489 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:36.490 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:53:36.494 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:53:36.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:53:37.002 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:37.486 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:37.951 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:37.952 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:53:44.515 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:53:44.515 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:53:45.022 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:45.956 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:53:51.527 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:53:52.014 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:52.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:53:52.034 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:53:52.455 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:52.456 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:53:52.542 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:53.015 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:53:53.473 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:53:58.535 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:53:59.522 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:53:59.523 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:53:59.552 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:53:59.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:54:00.060 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:00.511 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:54:01.104 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:01.106 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:54:07.202 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:54:07.710 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:54:08.163 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:08.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:54:08.217 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:54:08.630 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:08.631 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:09.176 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:54:14.715 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:54:15.710 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:15.711 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:54:15.730 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:54:16.155 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:54:16.239 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:16.694 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:54:23.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:54:23.244 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:54:23.656 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:54:23.752 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:24.193 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:24.194 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:54:29.744 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:54:29.758 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:54:30.268 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:30.736 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:30.738 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:54:31.192 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:31.193 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:54:37.700 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:54:37.714 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:54:38.226 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:38.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:38.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:54:39.145 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:39.146 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:54:45.223 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:54:45.652 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:45.653 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:54:45.729 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:46.204 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:46.689 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:47.151 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:47.152 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:54:53.199 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:54:53.233 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:54:53.653 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:53.740 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:54:54.182 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:54:54.183 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:55:01.250 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:55:01.759 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:55:02.251 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:02.252 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:55:02.267 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:55:02.708 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:02.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:55:02.775 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:03.851 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:55:10.955 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:55:11.462 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:55:11.942 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:11.943 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:55:11.970 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:55:12.396 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:12.397 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:55:12.477 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:12.942 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:55:13.390 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:55:19.959 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:55:19.979 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:55:20.423 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:55:20.485 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:20.965 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:20.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:21.410 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:21.411 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:55:26.978 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:55:27.484 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:55:27.979 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:27.983 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:55:27.991 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:55:28.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:28.434 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:55:28.496 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:28.961 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:29.414 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:55:34.286 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:55:34.793 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:55:35.300 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:35.301 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:55:35.302 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:55:35.742 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:35.743 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:55:35.809 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:36.299 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:36.792 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:37.248 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:37.249 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:55:44.838 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:55:45.320 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) , 2021-11-23T20:55:45.320 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2040 16 86688 86672 2.65501 0 - 0.0133532 , 2021-11-23T20:55:45.346 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:45.790 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:55:51.332 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:55:52.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:52.330 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:55:52.346 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:55:52.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:55:52.853 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:55:53.320 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2048 16 86688 86672 2.64464 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:53.339 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:55:53.787 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:55:53.788 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:55:54.320 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2049 16 86688 86672 2.64334 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:56:00.865 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:01.344 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:01.345 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:56:01.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:01.806 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:56:01.879 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:56:02.322 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2057 16 86688 86672 2.63306 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:02.808 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:56:06.998 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:56:07.506 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:56:07.972 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:07.973 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:56:08.013 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:56:08.430 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:08.431 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:08.974 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:56:14.503 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:56:15.508 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:15.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:56:15.517 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:56:15.964 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:56:16.024 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:16.519 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:16.965 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:16.967 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:56:23.521 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:56:23.527 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:56:24.033 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:24.529 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:24.531 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:56:24.976 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:24.977 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:56:25.049 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:56:30.530 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:56:31.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:31.038 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:56:31.039 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:56:31.492 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:31.493 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:56:31.546 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:32.015 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:56:39.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:56:39.151 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:39.637 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:40.081 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:40.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:56:45.586 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:56:45.646 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:46.141 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:46.142 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:46.591 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:46.592 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:56:53.151 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:56:53.659 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:56:54.131 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:54.132 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:56:54.166 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:56:54.581 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:56:54.582 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:56:55.135 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:56:55.586 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:57:01.364 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:57:02.835 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:57:02.886 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:57:03.331 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2118 16 86688 86672 2.55723 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:03.370 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:03.816 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:03.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:57:04.331 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2119 16 86688 86672 2.55602 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:57:09.344 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:57:09.380 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:57:09.888 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:10.346 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:10.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:57:10.811 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:10.812 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T20:57:11.332 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2126 16 86688 86672 2.54761 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:57:16.893 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:57:17.400 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:57:17.877 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:17.878 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:57:17.907 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:57:18.333 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:18.334 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:57:18.414 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:18.868 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:57:24.407 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:57:25.335 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2140 16 86688 86672 2.53094 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:25.406 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:25.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:25.858 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: 2021-11-23T20:57:26.912 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:26.913 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T20:57:27.019 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:57:34.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:34.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T20:57:34.542 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:35.030 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:35.485 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:57:42.530 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T20:57:42.562 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:57:43.068 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:43.533 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:43.534 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:57:49.568 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:57:50.540 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:50.541 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:57:50.583 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:57:50.995 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:51.078 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:51.525 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:57:58.167 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:57:58.170 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:57:58.610 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:57:58.678 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:59.159 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:59.161 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:57:59.612 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:57:59.613 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:58:00.166 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:58:06.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:58:06.453 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:58:06.905 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:06.906 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:58:13.439 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:58:13.457 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:58:13.968 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:58:14.442 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:14.443 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T20:58:14.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:14.896 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T20:58:14.983 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:58:20.973 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:58:21.482 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T20:58:21.959 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:21.960 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T20:58:21.990 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:58:22.407 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:22.408 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:58:22.958 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T20:58:23.412 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:58:29.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:58:29.502 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:58:30.009 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:58:30.467 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:30.468 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:58:36.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T20:58:36.688 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:58:37.198 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:58:37.703 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:37.704 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:58:38.148 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:38.149 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:58:38.212 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:58:44.193 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:58:44.699 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:44.700 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:58:45.152 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:45.153 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:58:45.207 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:58:53.219 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:58:53.718 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:53.719 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T20:58:54.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:58:54.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:58:54.235 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:58:55.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:59:00.727 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:59:01.720 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:01.721 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:59:01.741 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:59:02.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:59:02.249 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:02.697 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:59:07.491 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:59:08.485 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:08.486 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T20:59:08.505 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:59:08.933 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T20:59:09.012 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:09.481 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:09.937 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:09.938 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:59:15.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T20:59:16.019 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:59:16.449 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T20:59:16.525 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:16.974 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:16.975 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:59:24.036 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:24.523 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:24.524 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T20:59:24.980 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:24.981 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:59:25.050 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:59:31.525 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:31.526 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T20:59:31.538 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:59:31.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T20:59:32.044 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:32.520 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:32.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:32.984 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:59:39.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T20:59:39.336 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:59:39.759 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T20:59:39.842 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:40.292 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:40.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:40.762 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:40.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T20:59:46.338 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:46.845 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T20:59:47.303 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:47.304 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T20:59:47.352 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:48.289 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T20:59:53.840 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T20:59:54.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T20:59:54.806 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T20:59:54.854 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T20:59:55.265 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:55.355 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T20:59:55.811 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:00:01.860 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:00:02.312 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T21:00:02.358 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2297 16 86688 86672 2.35795 0 - 0.0133532 , 2021-11-23T21:00:02.366 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:00:02.774 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:02.860 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:03.323 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:03.324 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:03.818 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:03.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T21:00:04.358 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2299 16 86688 86672 2.3559 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:00:10.105 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:00:10.612 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:00:11.106 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:11.107 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T21:00:11.119 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:00:11.555 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:11.556 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:00:11.627 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:12.096 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:12.551 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:00:18.622 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:18.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T21:00:18.635 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:00:19.080 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:00:19.144 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:19.616 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:20.084 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:20.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:20.575 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:20.576 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T21:00:21.116 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:00:26.141 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:26.645 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:26.646 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T21:00:27.087 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:27.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T21:00:27.156 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T21:00:28.095 DEBUG:teuthology.orchestra.run:got remote process result: 22 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:00:33.142 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:00:33.648 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:00:34.125 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:34.127 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T21:00:34.154 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:00:34.579 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:34.580 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T21:00:34.662 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:35.112 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:00:42.271 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T21:00:42.297 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:00:42.716 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:42.804 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:43.267 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:43.268 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:43.718 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:43.718 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:00:48.773 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T21:00:48.790 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:00:49.297 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:49.764 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:49.765 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T21:00:50.211 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:50.212 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:00:56.813 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:00:57.297 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:57.298 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T21:00:57.322 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:00:57.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:00:57.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:00:57.830 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:00:58.285 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:01:04.820 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:04.822 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T21:01:04.835 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:01:05.280 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T21:01:05.343 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:01:12.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T21:01:12.501 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:01:13.007 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:13.513 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:13.514 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T21:01:13.952 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:13.953 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:01:14.021 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T21:01:14.529 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:15.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:15.465 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:01:22.021 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T21:01:22.045 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:01:22.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:01:22.554 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:23.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:23.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:23.481 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:23.482 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:01:29.047 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:29.545 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:29.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T21:01:30.003 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:30.005 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T21:01:30.064 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:31.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:01:36.558 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:36.559 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T21:01:36.563 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:01:37.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T21:01:37.069 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:37.561 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:38.019 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:01:43.854 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:01:44.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:44.853 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T21:01:44.868 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:01:45.306 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T21:01:45.373 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2021-11-23T21:01:45.374596+0000 min lat: 0.00170168 max lat: 3.50034 avg lat: 0.0133532 , 2021-11-23T21:01:45.373 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:01:51.837 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T21:01:51.870 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:01:52.374 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2407 16 86688 86672 2.25019 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:52.377 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:01:52.826 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:01:52.827 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:01:59.375 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2414 16 86688 86672 2.24367 0 - 0.0133532 , 2021-11-23T21:01:59.382 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:01:59.890 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:02:00.366 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:00.367 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:02:00.375 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2415 16 86688 86672 2.24274 0 - 0.0133532 Ignoring malparsed backtrace: 2021-11-23T21:02:00.814 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:00.816 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:01.369 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:01.830 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:02:06.389 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:02:06.894 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:06.895 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:02:06.896 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:02:07.341 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T21:02:07.376 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2422 16 86688 86672 2.23626 0 - 0.0133532 , 2021-11-23T21:02:07.403 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:07.889 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:08.340 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:08.342 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:08.850 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:08.851 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T21:02:09.377 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2424 16 86688 86672 2.23441 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:02:15.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T21:02:15.524 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:16.030 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:16.031 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:16.476 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:16.477 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:02:23.038 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:23.545 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:23.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T21:02:23.992 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:23.993 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T21:02:24.053 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:25.001 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:02:29.531 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:02:30.502 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:30.503 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T21:02:30.545 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:02:30.963 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:31.052 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:31.505 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:02:38.061 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:02:38.062 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:02:38.506 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T21:02:38.570 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:39.044 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:39.045 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:39.488 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:39.489 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:45.655 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:02:46.099 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:46.100 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T21:02:46.161 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:47.108 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:02:54.135 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T21:02:54.188 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:54.675 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:02:55.128 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:02:55.129 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:03:01.651 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T21:03:01.706 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:02.210 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:02.211 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:02.674 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:02.675 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:03:08.710 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:03:09.684 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:09.685 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T21:03:09.730 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:03:10.157 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T21:03:10.239 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:10.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:03:17.834 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T21:03:17.871 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:18.367 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:18.368 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:18.818 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:18.819 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:03:25.394 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:03:25.872 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:25.873 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:03:25.901 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:03:26.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T21:03:26.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2501 16 86688 86672 2.16562 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:26.860 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:27.326 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:27.327 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:03:34.423 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T21:03:34.435 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:03:34.944 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:35.406 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:35.407 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T21:03:35.859 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:35.860 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T21:03:36.390 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2511 16 86688 86672 2.15699 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:03:41.405 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T21:03:41.437 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:41.940 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:41.941 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:42.394 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:42.395 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:43.389 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:03:48.447 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:03:49.409 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:49.410 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T21:03:49.461 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:03:49.873 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:49.963 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: 2021-11-23T21:03:50.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2525 16 86688 86672 2.14503 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:50.417 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T21:03:51.393 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2526 16 86688 86672 2.14418 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:03:56.954 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:03:56.982 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:03:57.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:57.479 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:03:57.927 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:03:57.928 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:04:04.449 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T21:04:04.496 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:04.988 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:04.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:04:11.504 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:04:12.014 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:12.017 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T21:04:12.019 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:04:12.476 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:12.477 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T21:04:12.526 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:13.013 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:13.496 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:04:21.614 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:04:22.592 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:22.593 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T21:04:22.628 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:04:23.042 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:23.128 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:23.587 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:04:29.114 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T21:04:29.132 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:04:29.563 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:04:29.639 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:30.123 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:30.124 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:30.575 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:30.576 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:04:37.149 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:04:37.656 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:04:38.150 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:38.151 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:04:38.163 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:04:38.603 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:38.604 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T21:04:38.669 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:39.147 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:39.608 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:04:45.671 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T21:04:45.713 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:46.207 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:46.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:46.700 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:46.701 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T21:04:47.167 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:47.168 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T21:04:47.240 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:04:53.235 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:04:53.742 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:04:54.223 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:54.224 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:04:54.250 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:04:54.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:04:54.691 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T21:04:54.757 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:55.235 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:04:55.685 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:05:01.745 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:01.747 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:05:01.765 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:05:02.184 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:02.269 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:02.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:05:09.486 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:05:09.504 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:05:10.015 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:10.504 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:10.506 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T21:05:10.957 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:10.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T21:05:11.032 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T21:05:11.520 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:11.982 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:05:18.998 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T21:05:19.056 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:05:19.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:05:19.565 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:20.063 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:20.065 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:20.525 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:20.526 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:05:26.061 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:05:26.568 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:05:27.041 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:27.041 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T21:05:27.077 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:05:27.495 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:27.496 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T21:05:27.584 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:05:27.998 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:28.084 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:28.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:29.010 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:29.011 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:05:36.243 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:05:36.751 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:05:37.726 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:37.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:05:37.767 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:38.261 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:38.718 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:05:43.254 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:05:43.762 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:05:44.237 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:44.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:05:44.270 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:05:44.685 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:44.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:45.231 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:05:49.749 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:05:50.760 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:50.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:05:50.762 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:05:51.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T21:05:51.269 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:05:51.677 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:51.766 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:52.218 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:52.219 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:05:58.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T21:05:58.779 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:05:59.285 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:05:59.790 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:05:59.791 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T21:06:00.235 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:00.236 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T21:06:00.301 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:06:00.729 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:00.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:06:00.810 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:01.272 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:01.741 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:06:08.413 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2663 16 86688 86672 2.03387 0 - 0.0133532 , 2021-11-23T21:06:08.457 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:06:08.965 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:09.421 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:09.422 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T21:06:09.888 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:09.890 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T21:06:10.414 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2665 16 86688 86672 2.03235 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:06:15.966 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:06:16.474 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:06:16.942 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:16.943 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:06:16.982 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:06:17.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T21:06:17.414 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2672 16 86688 86672 2.02702 0 - 0.0133532 , 2021-11-23T21:06:17.489 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:06:17.903 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T21:06:17.997 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:06:18.415 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2673 16 86688 86672 2.02627 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:18.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:06:24.951 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T21:06:24.999 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:06:25.416 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2021-11-23T21:06:25.416588+0000 min lat: 0.00170168 max lat: 3.50034 avg lat: 0.0133532 , 2021-11-23T21:06:25.416 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) , 2021-11-23T21:06:25.416 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2680 16 86688 86672 2.02097 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:06:32.300 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:06:32.807 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:06:33.284 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:33.285 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T21:06:33.313 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:06:33.728 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:33.729 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight , 2021-11-23T21:06:33.819 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:34.291 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:34.798 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:35.246 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:35.247 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:06:41.782 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops , 2021-11-23T21:06:41.834 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:42.326 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:42.327 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:42.777 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:42.778 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T21:06:43.787 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:06:48.341 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:06:48.848 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:06:49.318 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:49.319 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T21:06:49.356 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:06:49.790 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:49.792 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T21:06:49.864 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:50.337 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:50.782 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:06:57.426 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:57.427 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T21:06:57.448 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:06:57.891 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T21:06:57.956 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:58.421 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2713 16 86688 86672 1.99639 0 - 0.0133532 , 2021-11-23T21:06:58.422 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:06:58.879 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:06:58.880 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T21:06:59.421 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2714 16 86688 86672 1.99566 0 - 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:07:05.469 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:07:05.970 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:05.971 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:07:05.977 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:07:06.429 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:06.430 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T21:07:06.484 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:06.966 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:07:13.941 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T21:07:13.984 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:07:14.424 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2729 16 86688 86672 1.98469 0 - 0.0133532 , 2021-11-23T21:07:14.490 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:14.951 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:14.952 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , 2021-11-23T21:07:15.423 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2730 16 86688 86672 1.98396 0 - 0.0133532 Ignoring malparsed backtrace: 2021-11-23T21:07:15.912 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:15.913 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , 2021-11-23T21:07:16.424 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2731 16 86688 86672 1.98323 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:07:22.000 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:07:22.507 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:07:22.982 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:22.983 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:07:23.013 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:07:23.443 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:23.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T21:07:23.521 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:23.989 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:24.446 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:07:30.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T21:07:30.080 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:07:30.492 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:30.587 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:31.033 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:31.034 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:07:38.101 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:38.608 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:38.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T21:07:39.073 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:39.075 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T21:07:39.118 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:40.061 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:07:45.103 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:07:45.512 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:45.513 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:46.052 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:07:53.116 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T21:07:53.126 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:07:53.572 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:07:53.634 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:54.110 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:54.112 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:07:54.563 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:07:54.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:08:01.698 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:08:02.677 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:02.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T21:08:02.712 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:08:03.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T21:08:03.220 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:03.701 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:04.144 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:04.145 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:04.647 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:04.648 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:08:10.245 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:10.750 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:10.751 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T21:08:11.212 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:11.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T21:08:11.263 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:12.223 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:08:17.776 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:18.283 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:08:18.298 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:08:18.735 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:18.736 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T21:08:18.806 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:19.285 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:19.735 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: 2021-11-23T21:08:25.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops , 2021-11-23T21:08:25.919 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:26.403 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:26.404 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:26.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:26.859 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:08:33.435 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2808 16 86688 86672 1.92885 0 - 0.0133532 , 2021-11-23T21:08:33.441 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:33.929 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:33.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: 2021-11-23T21:08:34.376 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:34.377 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T21:08:34.435 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2809 16 86688 86672 1.92816 0 - 0.0133532 Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:35.371 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:08:40.438 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2815 16 86688 86672 1.92405 0 - 0.0133532 , 2021-11-23T21:08:40.442 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:08:40.949 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:41.442 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:41.443 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T21:08:41.890 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:41.891 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:08:41.967 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:42.436 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:42.887 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:08:47.460 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:08:48.460 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:48.462 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:08:48.474 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:08:48.922 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T21:08:48.980 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:08:49.438 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2824 16 86688 86672 1.91792 0 - 0.0133532 , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:49.471 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:08:56.631 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , 2021-11-23T21:08:56.673 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:57.180 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:08:57.623 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:08:57.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:09:04.692 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:05.650 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:09:12.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T21:09:12.200 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:09:12.616 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:12.707 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:13.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:13.156 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:09:19.256 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T21:09:19.299 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:19.788 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:20.246 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:20.248 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:20.747 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:20.748 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:09:26.315 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:09:26.822 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:09:27.286 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:27.289 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:09:27.331 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:09:27.742 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:27.743 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:28.294 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:09:33.829 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:09:34.335 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:09:34.793 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:34.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:09:34.842 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:35.341 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:35.795 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:09:42.909 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T21:09:42.909 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:09:43.416 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:44.352 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:09:48.888 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , 2021-11-23T21:09:48.898 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:09:49.407 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:49.881 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:49.882 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T21:09:50.356 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:50.357 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops , 2021-11-23T21:09:50.422 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:09:56.421 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:09:56.929 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:09:57.397 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:57.399 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:09:57.436 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:09:57.854 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:09:57.855 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:09:58.394 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:10:03.932 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:10:04.913 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:04.914 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:10:04.946 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:10:05.366 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:05.449 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2021-11-23T21:10:05.449602+0000 min lat: 0.00170168 max lat: 3.50034 avg lat: 0.0133532 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:10:10.588 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:10:11.094 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:10:11.558 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:11.559 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:10:11.601 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:10:12.020 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:12.106 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:12.570 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:10:19.109 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:10:19.110 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:10:19.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T21:10:19.617 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:20.101 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:20.102 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:20.547 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:20.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:10:26.119 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:10:26.626 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:10:27.090 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:27.091 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T21:10:27.133 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:28.079 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:10:35.114 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:10:35.154 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:35.645 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:36.103 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:36.104 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:10:42.797 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:10:42.833 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:43.337 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:43.338 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:43.791 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:43.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:10:49.839 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:10:50.804 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:50.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T21:10:50.854 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:10:51.262 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:51.349 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:51.800 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:10:58.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T21:10:58.363 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:10:58.795 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T21:10:58.870 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:59.352 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:59.353 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:10:59.803 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:10:59.804 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: 2021-11-23T21:11:05.872 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:11:06.865 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:06.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , 2021-11-23T21:11:06.887 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:11:07.315 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight , 2021-11-23T21:11:07.394 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:07.866 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:11:16.043 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:16.547 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:16.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: 2021-11-23T21:11:17.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:17.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T21:11:17.056 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:17.979 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:11:25.060 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:11:25.070 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:25.486 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:25.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:11:31.565 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:11:31.569 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:11:32.076 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:32.551 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:32.552 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops Ignoring malparsed backtrace: 2021-11-23T21:11:33.012 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:33.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T21:11:33.093 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:11:39.184 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:11:39.692 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:11:40.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:40.172 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T21:11:40.201 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:11:40.635 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:40.636 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight , 2021-11-23T21:11:40.709 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:41.174 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:41.631 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:11:46.655 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight , 2021-11-23T21:11:46.697 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:11:47.204 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:47.655 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:47.656 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:11:54.161 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops , 2021-11-23T21:11:54.205 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:11:54.618 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:54.705 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:11:55.165 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:11:55.166 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:12:01.192 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops , 2021-11-23T21:12:01.210 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:12:01.640 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops , 2021-11-23T21:12:01.719 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:12:02.168 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:12:02.169 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: raise socket.error("Socket is closed") , OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:12:08.724 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: 2021-11-23T21:12:09.222 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:12:09.223 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight , 2021-11-23T21:12:09.231 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:12:09.668 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:12:09.669 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops , 2021-11-23T21:12:09.739 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:12:10.237 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:12:10.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: 2021-11-23T21:12:15.748 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:12:16.725 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:12:16.726 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops , 2021-11-23T21:12:16.762 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None Ignoring malparsed backtrace: 2021-11-23T21:12:17.181 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops , 2021-11-23T21:12:17.269 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:12:17.719 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send , raise socket.error("Socket is closed") , OSError: Socket is closed Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) Ignoring malparsed backtrace: 2021-11-23T21:12:24.226 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T21:12:24.279 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: OSError: Socket is closed , admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:12:24.764 DEBUG:teuthology.orchestra.run:got remote process result: 22 Ignoring malparsed backtrace: admin_socket: exception getting command descriptions: [Errno 111] Connection refused , 2021-11-23T21:12:25.214 DEBUG:teuthology.orchestra.run:got remote process result: 22 , 2021-11-23T21:12:25.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send , return self._send(s, m) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1198, in _send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall , sent = self.send(s) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 801, in send Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write , self.channel.sendall(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 846, in sendall Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all , count = self._write(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1364, in _write Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write , self._write_all(data) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 522, in _write_all Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal , self.proc.stdin.write(struct.pack('!b', sig)) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 405, in write Ignoring malparsed backtrace: 2021-11-23T21:12:30.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops , 2021-11-23T21:12:30.770 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): Ignoring malparsed backtrace: 2021-11-23T21:12:31.279 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None , Traceback (most recent call last): , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/orchestra/daemon/state.py", line 108, in signal Ignoring malparsed backtrace: manager = run_one_task(taskname, ctx=ctx, config=config) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/teuthology/run_tasks.py", line 70, in run_one_task , return task(**kwargs) Ignoring malparsed backtrace: 2021-11-23T21:13:44.481 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3119 16 86688 86672 1.73652 0 - 0.0133532 , 2021-11-23T21:13:45.482 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2021-11-23T21:13:45.482555+0000 min lat: 0.00170168 max lat: 3.50034 avg lat: 0.0133532 , 2021-11-23T21:13:45.482 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) Ignoring malparsed backtrace: 2021-11-23T21:15:09.494 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3204 16 86688 86672 1.69045 0 - 0.0133532 , 2021-11-23T21:15:10.494 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3205 16 86688 86672 1.68992 0 - 0.0133532 , 2021-11-23T21:15:11.494 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3206 16 86688 86672 1.6894 0 - 0.0133532 Ignoring malparsed backtrace: 2021-11-23T21:16:11.503 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3266 16 86688 86672 1.65836 0 - 0.0133532 , 2021-11-23T21:16:12.503 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3267 16 86688 86672 1.65785 0 - 0.0133532 , 2021-11-23T21:16:13.505 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3268 16 86688 86672 1.65735 0 - 0.0133532 Ignoring malparsed backtrace: 2021-11-23T21:17:44.517 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3359 16 86688 86672 1.61245 0 - 0.0133532 , 2021-11-23T21:17:45.517 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2021-11-23T21:17:45.518050+0000 min lat: 0.00170168 max lat: 3.50034 avg lat: 0.0133532 , 2021-11-23T21:17:45.518 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) Ignoring malparsed backtrace: next(self.gen) , File "/home/teuthworker/src/github.com_ceph_ceph-c_64d7747ea0581787abfa8e65c90131fc6a29c90f/qa/tasks/thrashosds.py", line 215, in task , cluster_manager.wait_for_all_osds_up() Ignoring malparsed backtrace: 2021-11-23T21:17:52.126 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.Q5k8bZlxd7 is 307KB , 2021-11-23T21:17:52.192 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.Q5k8bZlxd7 , 2021-11-23T21:17:52.200 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi/6522525/data/mon.c.tgz Ignoring malparsed backtrace: 2021-11-23T21:18:55.527 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3430 16 86688 86672 1.57907 0 - 0.0133532 , 2021-11-23T21:18:56.527 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3431 16 86688 86672 1.57861 0 - 0.0133532 , 2021-11-23T21:18:57.527 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3432 16 86688 86672 1.57815 0 - 0.0133532 Ignoring malparsed backtrace: node=self.hostname, label=self.label , teuthology.exceptions.CommandFailedError: Command failed on smithi103 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' ,