2020-11-18T14:41:32.783 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-18T14:41:32.784 INFO:teuthology.results:subset: '9000/10000' 2020-11-18T14:41:32.784 INFO:teuthology.results:seed: '3673' 2020-11-18T22:55:23.136 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-18T22:55:23.533 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-19T10:14:52.853 INFO:teuthology.results:Tests finished! gathering results... 2020-11-19T10:15:44.137 INFO:scrape:Found 446 jobs 2020-11-19T10:15:54.444 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6860-ge11b788b (e11b788b41a72cdbe2c77fab79a664262b358e1f) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14b) [0x7f58d384bb3d] , 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7f58d384bd18] 2020-11-19T10:15:54.447 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-18T22:32:54.355 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-18T22:32:54.659 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-19T10:15:54.449 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T22:34:57.843 INFO:teuthology.misc:Shutting down osd daemons... , 2020-11-18T22:34:57.843 DEBUG:tasks.ceph.osd.0:waiting for process to exit , 2020-11-18T22:34:57.844 INFO:teuthology.orchestra.run:waiting for 300 2020-11-19T10:15:54.450 WARNING:scrape:Ignoring malparsed backtrace: run.wait(tests.values()) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 470, in wait , proc.wait() 2020-11-19T10:15:54.627 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6860-ge11b788b (e11b788b41a72cdbe2c77fab79a664262b358e1f) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14b) [0x7f58d384bb3d] , 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7f58d384bd18] 2020-11-19T10:15:54.630 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-18T22:32:54.355 INFO:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-18T22:32:54.659 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-11-19T10:15:54.630 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T22:34:57.843 INFO:teuthology.misc:Shutting down osd daemons... , 2020-11-18T22:34:57.843 DEBUG:tasks.ceph.osd.0:waiting for process to exit , 2020-11-18T22:34:57.844 INFO:teuthology.orchestra.run:waiting for 300 2020-11-19T10:15:54.631 WARNING:scrape:Ignoring malparsed backtrace: run.wait(tests.values()) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 470, in wait , proc.wait() 2020-11-19T10:15:57.944 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6860-ge11b788b (e11b788b41a72cdbe2c77fab79a664262b358e1f) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f92bf0589b4] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5588197a4d56] 2020-11-19T10:15:57.945 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.261 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "waiting for peered", , 2020-11-18T16:25:54.261 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:08.872738+0000", , 2020-11-18T16:25:54.261 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 1.003641429 2020-11-19T10:15:57.946 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.284 INFO:teuthology.orchestra.run.smithi125.stdout: } , 2020-11-18T16:25:54.284 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.284 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.946 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.311 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.311 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "all_read", , 2020-11-18T16:25:54.311 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:08.870887+0000", 2020-11-19T10:15:57.947 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.341 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.341 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "reached_pg", , 2020-11-18T16:25:54.342 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:08.872873+0000", 2020-11-19T10:15:57.948 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.362 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.362 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "started", , 2020-11-18T16:25:54.362 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:09.898327+0000", 2020-11-19T10:15:57.948 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.382 INFO:teuthology.orchestra.run.smithi125.stdout: "initiated_at": "2020-11-18T16:24:08.871340+0000", , 2020-11-18T16:25:54.383 INFO:teuthology.orchestra.run.smithi125.stdout: "age": 105.37581254200001, , 2020-11-18T16:25:54.383 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 1.0312973620000001, 2020-11-19T10:15:57.949 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.403 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.403 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.403 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", 2020-11-19T10:15:57.949 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.423 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.423 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.424 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "started", 2020-11-19T10:15:57.950 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.443 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.443 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.444 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "done", 2020-11-19T10:15:57.951 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.463 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.00032150000000000001 , 2020-11-18T16:25:54.463 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.464 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.951 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.483 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 2.04e-06 , 2020-11-18T16:25:54.483 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.483 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.952 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.503 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.025280137000000001 , 2020-11-18T16:25:54.503 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.503 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.952 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.523 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 1.041599715 , 2020-11-18T16:25:54.523 INFO:teuthology.orchestra.run.smithi125.stdout: } , 2020-11-18T16:25:54.523 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2020-11-19T10:15:57.953 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.542 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:08.872522+0000", , 2020-11-18T16:25:54.543 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 2.491e-06 , 2020-11-18T16:25:54.543 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.954 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.562 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:33.815040+0000", , 2020-11-18T16:25:54.562 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.00014009499999999999 , 2020-11-18T16:25:54.563 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.954 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.582 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:34.720320+0000", , 2020-11-18T16:25:54.582 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 7.7503000000000007e-05 , 2020-11-18T16:25:54.582 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.955 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.037 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.037 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", , 2020-11-18T16:25:55.038 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:23:18.833511+0000", 2020-11-19T10:15:57.955 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.058 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.058 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.059 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "throttled", 2020-11-19T10:15:57.956 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.079 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.079 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.079 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "sub_op_commit_rec", 2020-11-19T10:15:57.957 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.099 INFO:teuthology.orchestra.run.smithi125.stdout: } , 2020-11-18T16:25:55.100 INFO:teuthology.orchestra.run.smithi125.stdout: ] , 2020-11-18T16:25:55.100 INFO:teuthology.orchestra.run.smithi125.stdout: } 2020-11-19T10:15:57.957 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.119 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0 , 2020-11-18T16:25:55.119 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.119 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.958 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.138 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 1.4005000000000001e-05 , 2020-11-18T16:25:55.138 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.139 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.958 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.158 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.001677879 , 2020-11-18T16:25:55.158 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.158 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.959 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.177 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.145203+0000", , 2020-11-18T16:25:55.177 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0 , 2020-11-18T16:25:55.177 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.960 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.196 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.145566+0000", , 2020-11-18T16:25:55.196 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.00083049600000000001 , 2020-11-18T16:25:55.197 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.960 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.216 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.099925669999999994, , 2020-11-18T16:25:55.216 INFO:teuthology.orchestra.run.smithi125.stdout: "type_data": { , 2020-11-18T16:25:55.216 INFO:teuthology.orchestra.run.smithi125.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-19T10:15:57.961 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.235 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", , 2020-11-18T16:25:55.235 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.270740+0000", , 2020-11-18T16:25:55.235 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 5.682e-06 2020-11-19T10:15:57.961 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.254 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "waiting for rw locks", , 2020-11-18T16:25:55.254 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.276011+0000", , 2020-11-18T16:25:55.254 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.087150330999999998 2020-11-19T10:15:57.962 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.273 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "sub_op_commit_rec", , 2020-11-18T16:25:55.273 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:24.418988+0000", , 2020-11-18T16:25:55.274 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 4.3296e-05 2020-11-19T10:15:57.962 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.293 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.293 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", , 2020-11-18T16:25:55.293 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:23.256861+0000", 2020-11-19T10:15:57.963 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.312 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.312 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "op_commit", , 2020-11-18T16:25:55.312 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:23.893091+0000", 2020-11-19T10:15:57.964 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.733 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.733 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.733 INFO:teuthology.orchestra.run.smithi125.stdout: "description": "osd_op(client.4513.0:2445 4.2 4:661bf87c:::smithi19740688-156:head [write 1438834~548839] snapc 0=[] ondisk+write+known_if_redirected e45)", 2020-11-19T10:15:57.964 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.753 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "queued_for_pg", , 2020-11-18T16:25:55.753 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:19.746452+0000", , 2020-11-18T16:25:55.753 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.000109156 2020-11-19T10:15:57.965 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.773 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.773 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "throttled", , 2020-11-18T16:25:55.774 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:19.746579+0000", 2020-11-19T10:15:57.965 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.793 INFO:teuthology.orchestra.run.smithi125.stdout: "events": [ , 2020-11-18T16:25:55.794 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.794 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "initiated", 2020-11-19T10:15:57.966 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.814 INFO:teuthology.orchestra.run.smithi125.stdout: } , 2020-11-18T16:25:55.814 INFO:teuthology.orchestra.run.smithi125.stdout: ] , 2020-11-18T16:25:55.814 INFO:teuthology.orchestra.run.smithi125.stdout: } 2020-11-19T10:15:57.967 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.834 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 2.875e-06 , 2020-11-18T16:25:55.835 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.835 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.967 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.855 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:58.725616+0000", , 2020-11-18T16:25:55.855 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 2.2993000000000002e-05 , 2020-11-18T16:25:55.855 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.968 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.875 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:58.725713+0000", , 2020-11-18T16:25:55.875 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.00056616100000000003 , 2020-11-18T16:25:55.875 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.968 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.895 INFO:teuthology.orchestra.run.smithi125.stdout: "client_addr": "172.21.15.197:0/166383830", , 2020-11-18T16:25:55.895 INFO:teuthology.orchestra.run.smithi125.stdout: "tid": 5854 , 2020-11-18T16:25:55.895 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.969 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.915 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "reached_pg", , 2020-11-18T16:25:55.915 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:18.292242+0000", , 2020-11-18T16:25:55.916 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.000131032 2020-11-19T10:15:57.969 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.935 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "done", , 2020-11-18T16:25:55.936 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:18.886846+0000", , 2020-11-18T16:25:55.936 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.60569063000000001 2020-11-19T10:15:57.970 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.956 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.956 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "header_read", , 2020-11-18T16:25:55.956 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:28.055972+0000", 2020-11-19T10:15:57.971 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.976 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.976 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "reached_pg", , 2020-11-18T16:25:55.976 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:28.057518+0000", 2020-11-19T10:15:57.971 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.396 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 3.5905999999999998e-05 , 2020-11-18T16:25:56.396 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:56.396 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.972 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.417 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:14.979532+0000", , 2020-11-18T16:25:56.417 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 5.5990000000000001e-06 , 2020-11-18T16:25:56.417 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.972 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "initiated", , 2020-11-18T16:25:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.089020+0000", , 2020-11-18T16:25:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0 2020-11-19T10:15:57.974 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "commit_sent", , 2020-11-18T16:25:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.149626+0000", , 2020-11-18T16:25:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 4.1721000000000003e-05 2020-11-19T10:15:57.975 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.480 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:56.480 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", , 2020-11-18T16:25:56.480 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.358350+0000", 2020-11-19T10:15:57.975 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.501 INFO:teuthology.orchestra.run.smithi125.stdout: "events": [ , 2020-11-18T16:25:56.501 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:56.501 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "initiated", 2020-11-19T10:15:57.976 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.522 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:56.522 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:56.522 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "commit_sent", 2020-11-19T10:15:57.977 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.542 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 7.2793999999999995e-05 , 2020-11-18T16:25:56.543 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:56.543 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:57.977 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.563 INFO:teuthology.orchestra.run.smithi125.stdout: "type_data": { , 2020-11-18T16:25:56.564 INFO:teuthology.orchestra.run.smithi125.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-11-18T16:25:56.564 INFO:teuthology.orchestra.run.smithi125.stdout: "events": [ 2020-11-19T10:15:57.978 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.585 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:32.692713+0000", , 2020-11-18T16:25:56.585 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.011037040999999999 , 2020-11-18T16:25:56.585 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:57.978 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.606 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "all_read", , 2020-11-18T16:25:56.606 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:39.587774+0000", , 2020-11-18T16:25:56.606 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 8.2477000000000003e-05 2020-11-19T10:15:57.979 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.627 INFO:teuthology.orchestra.run.smithi125.stdout: "age": 11.798551987, , 2020-11-18T16:25:56.627 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.018056341, , 2020-11-18T16:25:56.627 INFO:teuthology.orchestra.run.smithi125.stdout: "type_data": { 2020-11-19T10:15:57.979 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.060 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.060 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.060 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "header_read", 2020-11-19T10:15:57.980 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.081 INFO:teuthology.orchestra.run.smithi197.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-11-18T16:25:57.081 INFO:teuthology.orchestra.run.smithi197.stdout: "events": [ , 2020-11-18T16:25:57.081 INFO:teuthology.orchestra.run.smithi197.stdout: { 2020-11-19T10:15:57.981 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.101 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0.001502052 , 2020-11-18T16:25:57.102 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.102 INFO:teuthology.orchestra.run.smithi197.stdout: { 2020-11-19T10:15:57.981 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.122 INFO:teuthology.orchestra.run.smithi197.stdout: "time": "2020-11-18T16:24:14.394473+0000", , 2020-11-18T16:25:57.122 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0.13080459899999999 , 2020-11-18T16:25:57.122 INFO:teuthology.orchestra.run.smithi197.stdout: }, 2020-11-19T10:15:57.982 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.143 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0.083899778999999994, , 2020-11-18T16:25:57.143 INFO:teuthology.orchestra.run.smithi197.stdout: "type_data": { , 2020-11-18T16:25:57.143 INFO:teuthology.orchestra.run.smithi197.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-19T10:15:57.982 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.163 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "reached_pg", , 2020-11-18T16:25:57.163 INFO:teuthology.orchestra.run.smithi197.stdout: "time": "2020-11-18T16:24:17.910465+0000", , 2020-11-18T16:25:57.164 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0.086621658000000004 2020-11-19T10:15:57.983 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.184 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.184 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "dispatched", , 2020-11-18T16:25:57.184 INFO:teuthology.orchestra.run.smithi197.stdout: "time": "2020-11-18T16:24:18.197614+0000", 2020-11-19T10:15:57.983 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.204 INFO:teuthology.orchestra.run.smithi197.stdout: "client_info": { , 2020-11-18T16:25:57.204 INFO:teuthology.orchestra.run.smithi197.stdout: "client": "client.4513", , 2020-11-18T16:25:57.204 INFO:teuthology.orchestra.run.smithi197.stdout: "client_addr": "172.21.15.197:0/166383830", 2020-11-19T10:15:57.984 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.225 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.225 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.225 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "throttled", 2020-11-19T10:15:57.985 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.245 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.245 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.245 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "all_read", 2020-11-19T10:15:57.985 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.265 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.265 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.266 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "done", 2020-11-19T10:15:57.989 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.286 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 1.0113e-05 , 2020-11-18T16:25:57.286 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.286 INFO:teuthology.orchestra.run.smithi197.stdout: { 2020-11-19T10:15:57.990 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.306 INFO:teuthology.orchestra.run.smithi197.stdout: "time": "2020-11-18T16:25:08.390452+0000", , 2020-11-18T16:25:57.306 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0 , 2020-11-18T16:25:57.307 INFO:teuthology.orchestra.run.smithi197.stdout: }, 2020-11-19T10:15:57.992 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:26:10.970+0000 7f0cd8d6b700 -1 osd.4 103 handle_osd_map injecting map crc failure , reweighted osd.7 to 1 (10000) , 2020-11-18T16:26:11.091 INFO:teuthology.orchestra.run.smithi197:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2020-11-19T10:15:57.993 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:26:23.214 INFO:teuthology.orchestra.run.smithi197.stdout:} , 2020-11-18T16:26:23.225 INFO:teuthology.orchestra.run.smithi197:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 , 2020-11-18T16:26:23.547+0000 7f93b1837700 0 --2- 172.21.15.197:0/3645835526 >> [v2:172.21.15.197:3301/0,v1:172.21.15.197:6790/0] conn(0x7f93ac0a17b0 0x7f93ac137d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2020-11-19T10:15:57.995 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:26:47.225 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-11-18T16:25:21.286008+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=25, tm_sec=21, tm_wday=2, tm_yday=323, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=26, tm_sec=21, tm_wday=2, tm_yday=323, tm_isdst=0) , 2020-11-18T16:26:47.225 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2020-11-18T16:26:01.051457+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=26, tm_sec=1, tm_wday=2, tm_yday=323, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=26, tm_sec=21, tm_wday=2, tm_yday=323, tm_isdst=0) , 2020-11-18T16:26:47.226 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-11-18T16:25:18.242835+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=25, tm_sec=18, tm_wday=2, tm_yday=323, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=26, tm_sec=21, tm_wday=2, tm_yday=323, tm_isdst=0) 2020-11-19T10:15:57.995 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:27:28.823 INFO:tasks.ceph:Checking cluster log for badness... , 2020-11-18T16:27:28.824 INFO:teuthology.orchestra.run.smithi197:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(REQUEST_STUCK\)' | egrep -v 'failed to encode map' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'must scrub before tier agent can activate' | head -n 1 , 2020-11-18T16:27:28.882 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi197.front.sepia.ceph.com 2020-11-19T10:15:57.996 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:27:47.731 INFO:teuthology.orchestra.run.smithi197.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 1 +787us[ +787us] +/- 63ms , 2020-11-18T16:27:47.731 INFO:teuthology.orchestra.run.smithi125:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true , bash: ntpq: command not found 2020-11-19T10:15:57.996 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-19T10:15:58.107 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6860-ge11b788b (e11b788b41a72cdbe2c77fab79a664262b358e1f) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f92bf0589b4] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5588197a4d56] 2020-11-19T10:15:58.108 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.261 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "waiting for peered", , 2020-11-18T16:25:54.261 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:08.872738+0000", , 2020-11-18T16:25:54.261 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 1.003641429 2020-11-19T10:15:58.109 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.284 INFO:teuthology.orchestra.run.smithi125.stdout: } , 2020-11-18T16:25:54.284 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.284 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.109 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.311 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.311 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "all_read", , 2020-11-18T16:25:54.311 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:08.870887+0000", 2020-11-19T10:15:58.110 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.341 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.341 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "reached_pg", , 2020-11-18T16:25:54.342 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:08.872873+0000", 2020-11-19T10:15:58.111 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.362 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.362 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "started", , 2020-11-18T16:25:54.362 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:09.898327+0000", 2020-11-19T10:15:58.111 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.382 INFO:teuthology.orchestra.run.smithi125.stdout: "initiated_at": "2020-11-18T16:24:08.871340+0000", , 2020-11-18T16:25:54.383 INFO:teuthology.orchestra.run.smithi125.stdout: "age": 105.37581254200001, , 2020-11-18T16:25:54.383 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 1.0312973620000001, 2020-11-19T10:15:58.112 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.403 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.403 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.403 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", 2020-11-19T10:15:58.112 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.423 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.423 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.424 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "started", 2020-11-19T10:15:58.113 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.443 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.443 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:54.444 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "done", 2020-11-19T10:15:58.114 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.463 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.00032150000000000001 , 2020-11-18T16:25:54.463 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.464 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.114 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.483 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 2.04e-06 , 2020-11-18T16:25:54.483 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.483 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.115 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.503 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.025280137000000001 , 2020-11-18T16:25:54.503 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:54.503 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.115 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.523 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 1.041599715 , 2020-11-18T16:25:54.523 INFO:teuthology.orchestra.run.smithi125.stdout: } , 2020-11-18T16:25:54.523 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2020-11-19T10:15:58.116 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.542 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:08.872522+0000", , 2020-11-18T16:25:54.543 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 2.491e-06 , 2020-11-18T16:25:54.543 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.116 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.562 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:33.815040+0000", , 2020-11-18T16:25:54.562 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.00014009499999999999 , 2020-11-18T16:25:54.563 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.117 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:54.582 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:34.720320+0000", , 2020-11-18T16:25:54.582 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 7.7503000000000007e-05 , 2020-11-18T16:25:54.582 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.118 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.037 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.037 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", , 2020-11-18T16:25:55.038 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:23:18.833511+0000", 2020-11-19T10:15:58.118 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.058 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.058 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.059 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "throttled", 2020-11-19T10:15:58.119 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.079 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.079 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.079 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "sub_op_commit_rec", 2020-11-19T10:15:58.119 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.099 INFO:teuthology.orchestra.run.smithi125.stdout: } , 2020-11-18T16:25:55.100 INFO:teuthology.orchestra.run.smithi125.stdout: ] , 2020-11-18T16:25:55.100 INFO:teuthology.orchestra.run.smithi125.stdout: } 2020-11-19T10:15:58.120 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.119 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0 , 2020-11-18T16:25:55.119 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.119 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.121 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.138 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 1.4005000000000001e-05 , 2020-11-18T16:25:55.138 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.139 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.121 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.158 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.001677879 , 2020-11-18T16:25:55.158 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.158 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.122 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.177 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.145203+0000", , 2020-11-18T16:25:55.177 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0 , 2020-11-18T16:25:55.177 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.122 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.196 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.145566+0000", , 2020-11-18T16:25:55.196 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.00083049600000000001 , 2020-11-18T16:25:55.197 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.123 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.216 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.099925669999999994, , 2020-11-18T16:25:55.216 INFO:teuthology.orchestra.run.smithi125.stdout: "type_data": { , 2020-11-18T16:25:55.216 INFO:teuthology.orchestra.run.smithi125.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-19T10:15:58.123 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.235 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", , 2020-11-18T16:25:55.235 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.270740+0000", , 2020-11-18T16:25:55.235 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 5.682e-06 2020-11-19T10:15:58.124 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.254 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "waiting for rw locks", , 2020-11-18T16:25:55.254 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.276011+0000", , 2020-11-18T16:25:55.254 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.087150330999999998 2020-11-19T10:15:58.125 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.273 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "sub_op_commit_rec", , 2020-11-18T16:25:55.273 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:24.418988+0000", , 2020-11-18T16:25:55.274 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 4.3296e-05 2020-11-19T10:15:58.125 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.293 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.293 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", , 2020-11-18T16:25:55.293 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:23.256861+0000", 2020-11-19T10:15:58.126 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.312 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.312 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "op_commit", , 2020-11-18T16:25:55.312 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:23.893091+0000", 2020-11-19T10:15:58.126 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.733 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.733 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.733 INFO:teuthology.orchestra.run.smithi125.stdout: "description": "osd_op(client.4513.0:2445 4.2 4:661bf87c:::smithi19740688-156:head [write 1438834~548839] snapc 0=[] ondisk+write+known_if_redirected e45)", 2020-11-19T10:15:58.127 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.753 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "queued_for_pg", , 2020-11-18T16:25:55.753 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:19.746452+0000", , 2020-11-18T16:25:55.753 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.000109156 2020-11-19T10:15:58.127 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.773 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.773 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "throttled", , 2020-11-18T16:25:55.774 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:19.746579+0000", 2020-11-19T10:15:58.128 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.793 INFO:teuthology.orchestra.run.smithi125.stdout: "events": [ , 2020-11-18T16:25:55.794 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.794 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "initiated", 2020-11-19T10:15:58.129 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.814 INFO:teuthology.orchestra.run.smithi125.stdout: } , 2020-11-18T16:25:55.814 INFO:teuthology.orchestra.run.smithi125.stdout: ] , 2020-11-18T16:25:55.814 INFO:teuthology.orchestra.run.smithi125.stdout: } 2020-11-19T10:15:58.129 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.834 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 2.875e-06 , 2020-11-18T16:25:55.835 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:55.835 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.130 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.855 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:58.725616+0000", , 2020-11-18T16:25:55.855 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 2.2993000000000002e-05 , 2020-11-18T16:25:55.855 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.130 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.875 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:58.725713+0000", , 2020-11-18T16:25:55.875 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.00056616100000000003 , 2020-11-18T16:25:55.875 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.131 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.895 INFO:teuthology.orchestra.run.smithi125.stdout: "client_addr": "172.21.15.197:0/166383830", , 2020-11-18T16:25:55.895 INFO:teuthology.orchestra.run.smithi125.stdout: "tid": 5854 , 2020-11-18T16:25:55.895 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.131 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.915 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "reached_pg", , 2020-11-18T16:25:55.915 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:18.292242+0000", , 2020-11-18T16:25:55.916 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.000131032 2020-11-19T10:15:58.132 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.935 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "done", , 2020-11-18T16:25:55.936 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:18.886846+0000", , 2020-11-18T16:25:55.936 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.60569063000000001 2020-11-19T10:15:58.133 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.956 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.956 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "header_read", , 2020-11-18T16:25:55.956 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:28.055972+0000", 2020-11-19T10:15:58.133 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:55.976 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:55.976 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "reached_pg", , 2020-11-18T16:25:55.976 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:28.057518+0000", 2020-11-19T10:15:58.134 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.396 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 3.5905999999999998e-05 , 2020-11-18T16:25:56.396 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:56.396 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.134 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.417 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:14.979532+0000", , 2020-11-18T16:25:56.417 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 5.5990000000000001e-06 , 2020-11-18T16:25:56.417 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.135 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "initiated", , 2020-11-18T16:25:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.089020+0000", , 2020-11-18T16:25:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0 2020-11-19T10:15:58.135 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "commit_sent", , 2020-11-18T16:25:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.149626+0000", , 2020-11-18T16:25:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 4.1721000000000003e-05 2020-11-19T10:15:58.136 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.480 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:56.480 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "dispatched", , 2020-11-18T16:25:56.480 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:24:15.358350+0000", 2020-11-19T10:15:58.137 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.501 INFO:teuthology.orchestra.run.smithi125.stdout: "events": [ , 2020-11-18T16:25:56.501 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:56.501 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "initiated", 2020-11-19T10:15:58.137 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.522 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:56.522 INFO:teuthology.orchestra.run.smithi125.stdout: { , 2020-11-18T16:25:56.522 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "commit_sent", 2020-11-19T10:15:58.138 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.542 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 7.2793999999999995e-05 , 2020-11-18T16:25:56.543 INFO:teuthology.orchestra.run.smithi125.stdout: }, , 2020-11-18T16:25:56.543 INFO:teuthology.orchestra.run.smithi125.stdout: { 2020-11-19T10:15:58.138 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.563 INFO:teuthology.orchestra.run.smithi125.stdout: "type_data": { , 2020-11-18T16:25:56.564 INFO:teuthology.orchestra.run.smithi125.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-11-18T16:25:56.564 INFO:teuthology.orchestra.run.smithi125.stdout: "events": [ 2020-11-19T10:15:58.139 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.585 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:32.692713+0000", , 2020-11-18T16:25:56.585 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.011037040999999999 , 2020-11-18T16:25:56.585 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2020-11-19T10:15:58.139 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.606 INFO:teuthology.orchestra.run.smithi125.stdout: "event": "all_read", , 2020-11-18T16:25:56.606 INFO:teuthology.orchestra.run.smithi125.stdout: "time": "2020-11-18T16:25:39.587774+0000", , 2020-11-18T16:25:56.606 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 8.2477000000000003e-05 2020-11-19T10:15:58.140 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:56.627 INFO:teuthology.orchestra.run.smithi125.stdout: "age": 11.798551987, , 2020-11-18T16:25:56.627 INFO:teuthology.orchestra.run.smithi125.stdout: "duration": 0.018056341, , 2020-11-18T16:25:56.627 INFO:teuthology.orchestra.run.smithi125.stdout: "type_data": { 2020-11-19T10:15:58.141 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.060 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.060 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.060 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "header_read", 2020-11-19T10:15:58.141 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.081 INFO:teuthology.orchestra.run.smithi197.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-11-18T16:25:57.081 INFO:teuthology.orchestra.run.smithi197.stdout: "events": [ , 2020-11-18T16:25:57.081 INFO:teuthology.orchestra.run.smithi197.stdout: { 2020-11-19T10:15:58.142 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.101 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0.001502052 , 2020-11-18T16:25:57.102 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.102 INFO:teuthology.orchestra.run.smithi197.stdout: { 2020-11-19T10:15:58.142 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.122 INFO:teuthology.orchestra.run.smithi197.stdout: "time": "2020-11-18T16:24:14.394473+0000", , 2020-11-18T16:25:57.122 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0.13080459899999999 , 2020-11-18T16:25:57.122 INFO:teuthology.orchestra.run.smithi197.stdout: }, 2020-11-19T10:15:58.143 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.143 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0.083899778999999994, , 2020-11-18T16:25:57.143 INFO:teuthology.orchestra.run.smithi197.stdout: "type_data": { , 2020-11-18T16:25:57.143 INFO:teuthology.orchestra.run.smithi197.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-19T10:15:58.143 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.163 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "reached_pg", , 2020-11-18T16:25:57.163 INFO:teuthology.orchestra.run.smithi197.stdout: "time": "2020-11-18T16:24:17.910465+0000", , 2020-11-18T16:25:57.164 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0.086621658000000004 2020-11-19T10:15:58.144 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.184 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.184 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "dispatched", , 2020-11-18T16:25:57.184 INFO:teuthology.orchestra.run.smithi197.stdout: "time": "2020-11-18T16:24:18.197614+0000", 2020-11-19T10:15:58.145 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.204 INFO:teuthology.orchestra.run.smithi197.stdout: "client_info": { , 2020-11-18T16:25:57.204 INFO:teuthology.orchestra.run.smithi197.stdout: "client": "client.4513", , 2020-11-18T16:25:57.204 INFO:teuthology.orchestra.run.smithi197.stdout: "client_addr": "172.21.15.197:0/166383830", 2020-11-19T10:15:58.145 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.225 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.225 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.225 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "throttled", 2020-11-19T10:15:58.146 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.245 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.245 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.245 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "all_read", 2020-11-19T10:15:58.146 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.265 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.265 INFO:teuthology.orchestra.run.smithi197.stdout: { , 2020-11-18T16:25:57.266 INFO:teuthology.orchestra.run.smithi197.stdout: "event": "done", 2020-11-19T10:15:58.147 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.286 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 1.0113e-05 , 2020-11-18T16:25:57.286 INFO:teuthology.orchestra.run.smithi197.stdout: }, , 2020-11-18T16:25:57.286 INFO:teuthology.orchestra.run.smithi197.stdout: { 2020-11-19T10:15:58.148 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:25:57.306 INFO:teuthology.orchestra.run.smithi197.stdout: "time": "2020-11-18T16:25:08.390452+0000", , 2020-11-18T16:25:57.306 INFO:teuthology.orchestra.run.smithi197.stdout: "duration": 0 , 2020-11-18T16:25:57.307 INFO:teuthology.orchestra.run.smithi197.stdout: }, 2020-11-19T10:15:58.150 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:26:10.970+0000 7f0cd8d6b700 -1 osd.4 103 handle_osd_map injecting map crc failure , reweighted osd.7 to 1 (10000) , 2020-11-18T16:26:11.091 INFO:teuthology.orchestra.run.smithi197:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2020-11-19T10:15:58.151 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:26:23.214 INFO:teuthology.orchestra.run.smithi197.stdout:} , 2020-11-18T16:26:23.225 INFO:teuthology.orchestra.run.smithi197:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 , 2020-11-18T16:26:23.547+0000 7f93b1837700 0 --2- 172.21.15.197:0/3645835526 >> [v2:172.21.15.197:3301/0,v1:172.21.15.197:6790/0] conn(0x7f93ac0a17b0 0x7f93ac137d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2020-11-19T10:15:58.152 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:26:47.225 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-11-18T16:25:21.286008+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=25, tm_sec=21, tm_wday=2, tm_yday=323, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=26, tm_sec=21, tm_wday=2, tm_yday=323, tm_isdst=0) , 2020-11-18T16:26:47.225 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2020-11-18T16:26:01.051457+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=26, tm_sec=1, tm_wday=2, tm_yday=323, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=26, tm_sec=21, tm_wday=2, tm_yday=323, tm_isdst=0) , 2020-11-18T16:26:47.226 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-11-18T16:25:18.242835+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=25, tm_sec=18, tm_wday=2, tm_yday=323, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=18, tm_hour=16, tm_min=26, tm_sec=21, tm_wday=2, tm_yday=323, tm_isdst=0) 2020-11-19T10:15:58.153 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:27:28.823 INFO:tasks.ceph:Checking cluster log for badness... , 2020-11-18T16:27:28.824 INFO:teuthology.orchestra.run.smithi197:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(REQUEST_STUCK\)' | egrep -v 'failed to encode map' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'must scrub before tier agent can activate' | head -n 1 , 2020-11-18T16:27:28.882 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi197.front.sepia.ceph.com 2020-11-19T10:15:58.154 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-18T16:27:47.731 INFO:teuthology.orchestra.run.smithi197.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 1 +787us[ +787us] +/- 63ms , 2020-11-18T16:27:47.731 INFO:teuthology.orchestra.run.smithi125:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true , bash: ntpq: command not found 2020-11-19T10:15:58.154 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-19T10:15:58.537 WARNING:scrape:Misunderstood line: Binary file /home/teuthworker/archive/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627927/teuthology.log matches 2020-11-19T10:15:58.538 WARNING:scrape:Misunderstood line: 2020-11-19T10:15:58.816 INFO:scrape:Found 13 distinct failure reasons 2020-11-19T10:15:58.818 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi091 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2020-11-19T10:15:58.818 INFO:scrape:3 jobs: ['5628312', '5628101', '5628272'] 2020-11-19T10:15:58.819 INFO:scrape:suites intersection: ['rados'] 2020-11-19T10:15:58.819 INFO:scrape:suites union: ['ceph', 'clusters/9-mons', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/many', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/monthrash/{ceph', 'rados/verify/{centos_latest', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rados_api_tests', 'tasks/rados_api_tests}', 'thrashers/sync', 'validater/valgrind}', 'workloads/rados_api_tests}'] 2020-11-19T10:15:58.820 INFO:scrape: 2020-11-19T10:15:58.820 INFO:scrape:Missing packages 2020-11-19T10:15:58.821 INFO:scrape:2 jobs: ['5627857', '5627837'] 2020-11-19T10:15:58.821 INFO:scrape:suites intersection: ['ceph', 'mon_election/connectivity', 'rados', 'thrashers/pggrow', 'thrashosds-health'] 2020-11-19T10:15:58.821 INFO:scrape:suites union: ['1-install/mimic', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'clusters/{openstack', 'd-balancer/crush-compat', 'distro$/{ubuntu_18.04}', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/osd-delay', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{default}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/test_rbd_api}'] 2020-11-19T10:15:58.822 INFO:scrape: 2020-11-19T10:15:58.822 INFO:scrape:Failure: SSH connection to smithi133 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --balance-reads --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0' 2020-11-19T10:15:58.823 INFO:scrape:3 jobs: ['5628296', '5628216', '5627924'] 2020-11-19T10:15:58.823 INFO:scrape:suites intersection: ['rados'] 2020-11-19T10:15:58.824 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-async-recovery}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/many', 'msgr-failures/osd-delay', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'thrashers/mapgap', 'thrashers/pggrow', 'thrashosds-health', 'workloads/cache}', 'workloads/snaps-few-objects-balanced}'] 2020-11-19T10:15:58.824 INFO:scrape: 2020-11-19T10:15:58.824 INFO:scrape:Failure: {'Failure object was': {'smithi183.front.sepia.ceph.com': {'msg': 'Failed to update apt cache: ', 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'force-confdef,force-confold')"} 2020-11-19T10:15:58.825 INFO:scrape:8 jobs: ['5627849', '5627835', '5627836', '5627831', '5627843', '5627853', '5627830', '5627833'] 2020-11-19T10:15:58.825 INFO:scrape:suites intersection: [] 2020-11-19T10:15:58.826 INFO:scrape:suites union: ['clusters/{fixed-4', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/many', 'msgr/async-v1only', 'msgr/async-v2only', 'no_pools', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-stupid', 'openstack', 'openstack}', 'rados', 'rados/cephadm/smoke/{distro/ubuntu_18.04', 'rados/cephadm/smoke/{distro/ubuntu_18.04_podman', 'rados/multimon/{clusters/6', 'rados/perf/{ceph', 'rados/singleton-nomsgr/{all/recovery-unfound-found', 'rados/singleton/{all/erasure-code-nonregression', 'rados/standalone/{mon_election/connectivity', 'rados/thrash-erasure-code-shec/{ceph', 'recovery-overrides/{more-async-partial-recovery}', 'settings/optimized', 'start}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/mon_recovery}', 'thrashers/careful', 'thrashosds-health', 'ubuntu_latest', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'workloads/radosbench_4K_rand_read}', 'workloads/scrub}'] 2020-11-19T10:15:58.826 INFO:scrape: 2020-11-19T10:15:58.827 INFO:scrape:Failure: Command failed (workunit test cephadm/test_adoption.sh) on smithi174 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' 2020-11-19T10:15:58.827 INFO:scrape:3 jobs: ['5627858', '5628235', '5628032'] 2020-11-19T10:15:58.828 INFO:scrape:suites intersection: ['task/test_adoption}'] 2020-11-19T10:15:58.828 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/centos_latest', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2020-11-19T10:15:58.828 INFO:scrape: 2020-11-19T10:15:58.829 INFO:scrape:Failure: Command failed on smithi105 with status 5: 'sudo systemctl stop ceph-09bf34b2-29c6-11eb-b4fd-001a4aab830c@mon.a' 2020-11-19T10:15:58.829 INFO:scrape:7 jobs: ['5628046', '5627941', '5628204', '5628308', '5627900', '5628259', '5628114'] 2020-11-19T10:15:58.830 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', 'fixed-2', 'rados/cephadm/upgrade/{1-start'] 2020-11-19T10:15:58.830 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{centos_8.0}', 'distro$/{centos_latest}', 'distro$/{rhel_latest}', 'distro$/{ubuntu_18.04_podman}', 'distro$/{ubuntu_18.04}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2020-11-19T10:15:58.830 INFO:scrape: 2020-11-19T10:15:58.831 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2020-11-19T10:15:58.831 INFO:scrape:3 jobs: ['5627911', '5628066', '5628280'] 2020-11-19T10:15:58.832 INFO:scrape:suites intersection: ['task/test_cephadm}'] 2020-11-19T10:15:58.832 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/centos_latest', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2020-11-19T10:15:58.832 INFO:scrape: 2020-11-19T10:15:58.833 INFO:scrape:Failure: '/home/ubuntu/cephtest/archive/syslog/misc.log:2020-11-18T17:18:40.634669+00:00 smithi188 bash[7319]: debug 2020-11-18T17:18:40.632+0000 7fab697d5700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr:INFO:cephadm:Deploying daemon prometheus.a ... ' in syslog 2020-11-19T10:15:58.833 INFO:scrape:1 jobs: ['5627998'] 2020-11-19T10:15:58.834 INFO:scrape:suites: ['2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{ubuntu_latest}', 'fixed-2', 'mon_election/classic}', 'rados/cephadm/upgrade/{1-start'] 2020-11-19T10:15:58.834 INFO:scrape: 2020-11-19T10:15:58.836 INFO:scrape:Dead: b'2020-11-19T10:12:26.131 DEBUG:teuthology.exit:Finished running handlers' 2020-11-19T10:15:58.836 INFO:scrape:2 jobs: ['5628212', '5627978'] 2020-11-19T10:15:58.836 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'mon_election/classic', 'openstack}', 'rados', 'supported-random-distro$/{rhel_8}', 'thrashosds-health'] 2020-11-19T10:15:58.837 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/crush-compat', 'fast/fast', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'recovery-overrides/{more-async-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/careful', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=2-m=1}', 'workloads/write_fadvise_dontneed}'] 2020-11-19T10:15:58.837 INFO:scrape: 2020-11-19T10:15:58.838 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/mon/caps.sh 2020-11-19T10:15:58.838 INFO:scrape:1 jobs: ['5628021'] 2020-11-19T10:15:58.839 INFO:scrape:suites: ['clusters/3-mons', 'mon_election/connectivity', 'msgr-failures/mon-delay', 'msgr/async-v2only', 'objectstore/bluestore-stupid', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/sync', 'workloads/rados_mon_workunits}'] 2020-11-19T10:15:58.839 INFO:scrape: 2020-11-19T10:15:58.839 INFO:scrape:Failure: Found coredumps on ubuntu@smithi075.front.sepia.ceph.com 2020-11-19T10:15:58.840 INFO:scrape:1 jobs: ['5628291'] 2020-11-19T10:15:58.840 INFO:scrape:suites: ['fixed-2', 'mon_election/connectivity', 'rados/cephadm/smoke/{distro/rhel_8.0', 'start}'] 2020-11-19T10:15:58.840 INFO:scrape: 2020-11-19T10:15:58.841 INFO:scrape:Failure: "2020-11-18T15:06:46.046134+0000 osd.3 (osd.3) 339 : cluster [ERR] scrub 59.1f 59:faf90f2b:test-rados-api-smithi189-43050-74::foo:24 : size 0 != clone_size 10" in cluster log 2020-11-19T10:15:58.841 INFO:scrape:1 jobs: ['5627834'] 2020-11-19T10:15:58.842 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-partial-recovery}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/osd-delay', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{rhel_8}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/rados_api_tests}'] 2020-11-19T10:15:58.842 INFO:scrape: 2020-11-19T10:15:58.843 INFO:scrape:Valgrind: 2020-11-19T10:15:58.843 INFO:scrape:1 jobs: ['5627927'] 2020-11-19T10:15:58.843 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/mon_recovery', 'validater/valgrind}'] 2020-11-19T10:15:58.844 INFO:scrape: 2020-11-19T10:15:58.859 INFO:teuthology.results:Sending results to ksirivad@redhat.com: Test Run: ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/ failed: 26 dead: 10 running: 0 waiting: 0 queued: 0 passed: 410 Fail ================================================================= [5627834] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:26:54 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627834/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627834/ "2020-11-18T15:06:46.046134+0000 osd.3 (osd.3) 339 : cluster [ERR] scrub 59.1f 59:faf90f2b:test-rados-api-smithi189-43050-74::foo:24 : size 0 != clone_size 10" in cluster log [5627837] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627837/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627837/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e4d3c929f40a49c393a6107ef11a6b92 Command failed on smithi134 with status 100: 'sudo apt-get update' [5627857] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627857/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627857/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9ef1de17d1a940f1ac718b7597c0bfdd Command failed on smithi033 with status 100: 'sudo apt-get update' [5627858] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:12:12 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627858/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627858/ sentry: https://sentry.ceph.com/organizations/ceph/?query=afff5a88b795498eaaf30e43e2c331e0 Command failed (workunit test cephadm/test_adoption.sh) on smithi174 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5627900] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:02:02 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627900/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627900/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4978575a1c9a48508510b302cb28df65 Command failed on smithi065 with status 5: 'sudo systemctl stop ceph-0ad98a1e-29b7-11eb-b4fd-001a4aab830c@mon.a' [5627911] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:05:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627911/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627911/ sentry: https://sentry.ceph.com/organizations/ceph/?query=40851510b6964a309de4694208a51994 Command failed (workunit test cephadm/test_cephadm.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5627924] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache} ----------------------------------------------------------------- time: 00:14:54 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627924/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627924/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op cache_flush 50 --op cache_try_flush 50 --op cache_evict 50 --op write_excl 50 --pool base' [5627927] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 00:24:57 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627927/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627927/ saw valgrind issues [5627941] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_latest} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:07:03 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627941/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627941/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d7d36343df5f4126963cbba76022dd39 Command failed on smithi026 with status 5: 'sudo systemctl stop ceph-8303a1b4-29bc-11eb-b4fd-001a4aab830c@mon.a' [5627998] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 01:10:03 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627998/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627998/ '/home/ubuntu/cephtest/archive/syslog/misc.log:2020-11-18T17:18:40.634669+0 0:00 smithi188 bash[7319]: debug 2020-11-18T17:18:40.632+0000 7fab697d5700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr:INFO:cephadm:Deploying daemon prometheus.a ... ' in syslog [5628021] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_workunits} ----------------------------------------------------------------- time: 03:13:05 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628021/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628021/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e8e45c0479c459586ed675eb0eef16c Command failed (workunit test mon/caps.sh) on smithi047 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh' [5628032] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:04:50 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628032/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628032/ sentry: https://sentry.ceph.com/organizations/ceph/?query=71c682927c4f4635aab375ffc62baac6 Command failed (workunit test cephadm/test_adoption.sh) on smithi193 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5628046] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04_podman} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:02:07 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628046/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628046/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d661e44118114160a81b82139b88a3fc Command failed on smithi105 with status 5: 'sudo systemctl stop ceph-09bf34b2-29c6-11eb-b4fd-001a4aab830c@mon.a' [5628066] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:05:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628066/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628066/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2f4a0021b18d44eca7cbc7a5217b24c4 Command failed (workunit test cephadm/test_cephadm.sh) on smithi050 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5628101] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:42:29 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628101/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628101/ sentry: https://sentry.ceph.com/organizations/ceph/?query=900056a3bc4747388088b28016a00d67 Command failed (workunit test rados/test.sh) on smithi166 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5628114] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:04:24 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628114/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628114/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6687a39dcf1a496cad333d605cd68559 Command failed on smithi038 with status 5: 'sudo systemctl stop ceph-31db33dc-29d1-11eb-b4fd-001a4aab830c@mon.a' [5628204] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:04:27 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628204/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628204/ sentry: https://sentry.ceph.com/organizations/ceph/?query=644c9bcffd08483f88d04cfaf8a45854 Command failed on smithi139 with status 5: 'sudo systemctl stop ceph-58b951c6-29ec-11eb-b4fd-001a4aab830c@mon.a' [5628216] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:15:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628216/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628216/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [5628235] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:05:27 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628235/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628235/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9ba57e74bc07492182acaa1b0d031a02 Command failed (workunit test cephadm/test_adoption.sh) on smithi109 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5628259] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:02:50 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628259/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628259/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b67756951d3243a88c1836edb38733c2 Command failed on smithi063 with status 5: 'sudo systemctl stop ceph-55a55b44-29ef-11eb-b4fd-001a4aab830c@mon.a' [5628272] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:26:30 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628272/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628272/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c00bda4ab2ca4a01985169a89b476c56 Command failed (workunit test rados/test.sh) on smithi184 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5628280] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:05:04 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628280/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628280/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2f73a37775444755b90e5236d2a20062 Command failed (workunit test cephadm/test_cephadm.sh) on smithi029 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5628291] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:15:37 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628291/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628291/ Found coredumps on ubuntu@smithi075.front.sepia.ceph.com [5628296] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} ----------------------------------------------------------------- time: 00:21:52 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628296/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628296/ SSH connection to smithi133 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --balance-reads --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0' [5628308] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04_podman} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:02:03 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628308/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628308/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f9940084a772419c93988aa500e160fb Command failed on smithi195 with status 5: 'sudo systemctl stop ceph-8225f496-29f2-11eb-b4fd-001a4aab830c@mon.a' [5628312] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:19:02 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628312/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628312/ sentry: https://sentry.ceph.com/organizations/ceph/?query=13683ced74544bd694e569663e556a3d Command failed (workunit test rados/test.sh) on smithi091 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e11b788b41a72cdbe2c77fab79a664262b358e1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' Dead ================================================================= [5627830] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:12:56 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627830/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627830/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6025ba8fc0c447fa8cc0bf897744e675 {'Failure object was': {'smithi065.front.sepia.ceph.com': {'msg': "Failed to update apt cache: E:The repository 'http://security.ubuntu.com/ubuntu bionic-security Release' no longer has a Release file.", 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te uthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'force-confdef,force-confold')"} [5627831] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} ----------------------------------------------------------------- time: 00:13:14 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627831/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627831/ sentry: https://sentry.ceph.com/organizations/ceph/?query=67c18b52a9ed48519107c537b1414c56 {'Failure object was': {'smithi174.front.sepia.ceph.com': {'msg': 'Failed to update apt cache: ', 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te uthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'force-confdef,force-confold')"} [5627833] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} ----------------------------------------------------------------- time: 00:13:17 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627833/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627833/ sentry: https://sentry.ceph.com/organizations/ceph/?query=03a4abe5531f4afebe260144bb77e9db {'Failure object was': {'smithi067.front.sepia.ceph.com': {'msg': 'Failed to update apt cache: ', 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te uthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'force-confdef,force-confold')"} [5627835] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:12:24 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627835/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627835/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fd23a9c808aa4453aa204f7a3e9ef7b0 {'Failure object was': {'smithi162.front.sepia.ceph.com': {'msg': "Failed to update apt cache: E:The repository 'http://security.ubuntu.com/ubuntu bionic-security Release' no longer has a Release file.", 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te uthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'force-confdef,force-confold')"} [5627836] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/scrub} ----------------------------------------------------------------- time: 00:11:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627836/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627836/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6ae085836b9946239fd8cc26366d2c67 {'Failure object was': {'smithi170.front.sepia.ceph.com': {'msg': 'Failed to update apt cache: ', 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te uthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'force-confdef,force-confold')"} [5627843] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:11:48 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627843/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627843/ sentry: https://sentry.ceph.com/organizations/ceph/?query=77646098737f4489ae448aaf6599c008 {'Failure object was': {'smithi112.front.sepia.ceph.com': {'msg': 'Failed to update apt cache: ', 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te uthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'force-confdef,force-confold')"} [5627849] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:13:30 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627849/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627849/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7da5596f978644a4b66f56fe74608823 {'Failure object was': {'smithi183.front.sepia.ceph.com': {'msg': 'Failed to update apt cache: ', 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te uthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'force-confdef,force-confold')"} [5627853] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:11:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627853/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627853/ sentry: https://sentry.ceph.com/organizations/ceph/?query=174e15d951e64e9e99492c2c9dc279cf {'Failure object was': {'smithi043.front.sepia.ceph.com': {'msg': 'Failed to update apt cache: ', 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te uthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'force-confdef,force-confold')"} [5627978] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627978/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627978/ [5628212] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628212/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628212/ Pass ================================================================= [5627829] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:34:02 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627829/ [5627832] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:14:32 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627832/ [5627838] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:57 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627838/ [5627839] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:42:54 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627839/ [5627840] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:46:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627840/ [5627841] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:18:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627841/ [5627842] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:50:46 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627842/ [5627844] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/radosbench} time: 01:44:16 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627844/ [5627845] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:10 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627845/ [5627846] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/prometheus} time: 00:08:54 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627846/ [5627847] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/pool-create-delete} time: 00:13:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627847/ [5627848] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:28:19 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627848/ [5627850] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:21:05 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627850/ [5627851] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:17:17 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627851/ [5627852] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:18:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627852/ [5627854] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:41:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627854/ [5627855] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:11:40 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627855/ [5627856] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:10:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627856/ [5627859] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:12:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627859/ [5627860] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:11:46 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627860/ [5627861] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:18:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627861/ [5627862] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:24:08 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627862/ [5627863] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 01:00:06 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627863/ [5627864] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627864/ [5627865] rados/objectstore/{backends/alloc-hint supported-random-distro$/{centos_8}} time: 00:06:17 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627865/ [5627866] rados/rest/{mgr-restful supported-random-distro$/{rhel_8}} time: 00:12:09 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627866/ [5627867] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:13:39 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627867/ [5627868] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/crush} time: 00:13:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627868/ [5627869] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_latest} time: 05:22:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627869/ [5627870] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:15:40 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627870/ [5627871] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:35:54 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627871/ [5627872] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:17:35 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627872/ [5627873] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:21:20 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627873/ [5627874] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:20:20 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627874/ [5627875] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/radosbench} time: 01:38:35 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627875/ [5627876] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627876/ [5627877] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:27:15 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627877/ [5627878] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} time: 00:13:59 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627878/ [5627879] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:10:58 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627879/ [5627880] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:59:01 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627880/ [5627881] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:30:39 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627881/ [5627882] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:13 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627882/ [5627883] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:14:56 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627883/ [5627884] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:40 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627884/ [5627885] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{rhel_8}} time: 00:15:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627885/ [5627886] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:28:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627886/ [5627887] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:12:38 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627887/ [5627888] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:05:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627888/ [5627889] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:08:57 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627889/ [5627890] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:26:02 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627890/ [5627891] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_5925} time: 00:06:26 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627891/ [5627892] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:22:13 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627892/ [5627893] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:12:40 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627893/ [5627894] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:14:01 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627894/ [5627895] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:05:24 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627895/ [5627896] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:27:14 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627896/ [5627897] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:10:15 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627897/ [5627898] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:41:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627898/ [5627899] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:11:24 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627899/ [5627901] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:19:26 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627901/ [5627902] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:13:59 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627902/ [5627903] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:34:33 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627903/ [5627904] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:16:07 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627904/ [5627905] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627905/ [5627906] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:24:32 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627906/ [5627907] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:09:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627907/ [5627908] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:20:16 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627908/ [5627909] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:11:06 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627909/ [5627910] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:19:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627910/ [5627912] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:30:23 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627912/ [5627913] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:21:20 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627913/ [5627914] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:29:12 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627914/ [5627915] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:09:06 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627915/ [5627916] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:09:11 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627916/ [5627917] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:40 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627917/ [5627918] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/dashboard} time: 01:04:27 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627918/ [5627919] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} time: 00:09:08 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627919/ [5627920] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:52 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627920/ [5627921] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:23:37 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627921/ [5627922] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:04:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627922/ [5627923] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:11:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627923/ [5627925] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:05:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627925/ [5627926] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627926/ [5627928] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:45:03 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627928/ [5627929] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects} time: 00:25:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627929/ [5627930] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/crash} time: 00:13:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627930/ [5627931] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:12:15 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627931/ [5627932] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/rados_api_tests} time: 00:21:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627932/ [5627933] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:14:53 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627933/ [5627934] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_api_tests} time: 00:30:54 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627934/ [5627935] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:28:50 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627935/ [5627936] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/mgr} time: 00:14:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627936/ [5627937] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:05:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627937/ [5627938] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency} time: 00:39:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627938/ [5627939] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:32:14 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627939/ [5627940] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench} time: 02:15:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627940/ [5627942] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:07:19 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627942/ [5627943] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:06:48 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627943/ [5627944] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:08:03 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627944/ [5627945] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect} time: 00:09:41 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627945/ [5627946] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm_repos} time: 00:03:19 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627946/ [5627947] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_python} time: 00:17:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627947/ [5627948] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:39 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627948/ [5627949] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:07 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627949/ [5627950] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect_promote_tests} time: 00:14:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627950/ [5627951] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:15:23 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627951/ [5627952] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:20:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627952/ [5627953] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect_set_object} time: 00:12:24 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627953/ [5627954] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:47 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627954/ [5627955] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:16:26 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627955/ [5627956] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/set-chunks-read} time: 00:16:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627956/ [5627957] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:15:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627957/ [5627958] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627958/ [5627959] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/small-objects-balanced} time: 00:19:37 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627959/ [5627960] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:07:01 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627960/ [5627961] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:13:33 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627961/ [5627962] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/misc} time: 00:29:15 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627962/ [5627963] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} time: 00:19:03 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627963/ [5627964] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:21:02 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627964/ [5627965] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:06:39 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627965/ [5627966] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:12:52 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627966/ [5627967] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/dashboard} time: 01:08:15 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627967/ [5627968] rados/objectstore/{backends/filejournal supported-random-distro$/{ubuntu_latest}} time: 00:05:04 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627968/ [5627969] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:26:05 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627969/ [5627970] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects} time: 00:16:11 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627970/ [5627971] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:35:37 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627971/ [5627972] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:20:23 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627972/ [5627973] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:35:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627973/ [5627974] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:09:41 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627974/ [5627975] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:28:53 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627975/ [5627976] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:11:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627976/ [5627977] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/rados_mon_osdmap_prune} time: 00:38:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627977/ [5627979] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:13:07 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627979/ [5627980] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} time: 00:24:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627980/ [5627981] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:16:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627981/ [5627982] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:15:05 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627982/ [5627983] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:26:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627983/ [5627984] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/rados_stress_watch} time: 00:20:07 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627984/ [5627985] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} time: 00:15:38 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627985/ [5627986] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed} time: 00:12:33 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627986/ [5627987] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:06:51 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627987/ [5627988] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:19:51 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627988/ [5627989] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:31:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627989/ [5627990] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627990/ [5627991] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:23:26 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627991/ [5627992] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:28 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627992/ [5627993] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-agent-big} time: 00:24:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627993/ [5627994] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:12:32 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627994/ [5627995] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/sample_fio} time: 00:10:53 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627995/ [5627996] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/mon} time: 00:37:55 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627996/ [5627997] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-small} time: 00:11:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627997/ [5627999] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5627999/ [5628000] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:33 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628000/ [5628001] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:22:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628001/ [5628002] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:20:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628002/ [5628003] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:28:08 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628003/ [5628004] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:11:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628004/ [5628005] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:24:11 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628005/ [5628006] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:10:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628006/ [5628007] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:23:13 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628007/ [5628008] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:10:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628008/ [5628009] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:07 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628009/ [5628010] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:10:53 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628010/ [5628011] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:07:58 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628011/ [5628012] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:02:28 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628012/ [5628013] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:25:11 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628013/ [5628014] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/insights} time: 00:16:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628014/ [5628015] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:22:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628015/ [5628016] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:21:13 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628016/ [5628017] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/dashboard} time: 01:07:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628017/ [5628018] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} time: 02:09:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628018/ [5628019] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:18:20 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628019/ [5628020] rados/singleton/{all/peer mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:11:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628020/ [5628022] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:33:46 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628022/ [5628023] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache} time: 00:08:10 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628023/ [5628024] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_striper} time: 00:11:53 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628024/ [5628025] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:29:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628025/ [5628026] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628026/ [5628027] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:29:06 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628027/ [5628028] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628028/ [5628029] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:19:06 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628029/ [5628030] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/rados_api_tests} time: 00:21:49 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628030/ [5628031] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:07:13 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628031/ [5628033] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:13:32 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628033/ [5628034] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:54:04 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628034/ [5628035] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:39:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628035/ [5628036] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:19 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628036/ [5628037] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/radosbench} time: 01:56:15 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628037/ [5628038] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:14:55 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628038/ [5628039] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628039/ [5628040] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:09:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628040/ [5628041] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:05:07 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628041/ [5628042] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628042/ [5628043] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:05:33 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628043/ [5628044] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:11:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628044/ [5628045] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:09:04 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628045/ [5628047] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:11:01 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628047/ [5628048] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} time: 00:15:35 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628048/ [5628049] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/set-chunks-read} time: 00:12:50 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628049/ [5628050] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:20:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628050/ [5628051] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:11:58 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628051/ [5628052] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:12:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628052/ [5628053] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:28:30 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628053/ [5628054] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:10:59 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628054/ [5628055] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:25:33 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628055/ [5628056] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} time: 00:24:56 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628056/ [5628057] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:26:38 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628057/ [5628058] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:24:02 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628058/ [5628059] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:09:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628059/ [5628060] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:30:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628060/ [5628061] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects-localized} time: 00:19:51 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628061/ [5628062] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:47 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628062/ [5628063] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/snaps-few-objects} time: 00:26:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628063/ [5628064] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 02:17:28 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628064/ [5628065] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/small-objects} time: 00:25:41 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628065/ [5628067] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:31:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628067/ [5628068] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628068/ [5628069] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{rhel_8}} time: 02:23:33 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628069/ [5628070] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:31:12 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628070/ [5628071] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} time: 00:26:46 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628071/ [5628072] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:14:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628072/ [5628073] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:26:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628073/ [5628074] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628074/ [5628075] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/cosbench_64K_write} time: 00:19:41 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628075/ [5628076] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:25:30 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628076/ [5628077] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:06 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628077/ [5628078] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:11:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628078/ [5628079] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:26:16 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628079/ [5628080] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:26:06 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628080/ [5628081] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:11:49 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628081/ [5628082] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:08 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628082/ [5628083] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:34:29 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628083/ [5628084] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:02 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628084/ [5628085] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} time: 00:22:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628085/ [5628086] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:12:46 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628086/ [5628087] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:18:35 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628087/ [5628088] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/dashboard} time: 01:04:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628088/ [5628089] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:19 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628089/ [5628090] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/crush} time: 00:08:28 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628090/ [5628091] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_latest} time: 05:11:32 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628091/ [5628092] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mix} time: 00:22:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628092/ [5628093] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:11:48 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628093/ [5628094] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:16:14 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628094/ [5628095] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-agent-small} time: 00:15:20 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628095/ [5628096] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} time: 00:14:02 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628096/ [5628097] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:31:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628097/ [5628098] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/progress} time: 00:17:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628098/ [5628099] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:17:38 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628099/ [5628100] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:05 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628100/ [5628102] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{centos_8}} time: 00:10:01 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628102/ [5628103] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-pool-snaps} time: 00:26:51 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628103/ [5628104] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:10:04 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628104/ [5628105] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/many workloads/snaps-few-objects} time: 00:32:09 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628105/ [5628106] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:21:01 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628106/ [5628107] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:27 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628107/ [5628108] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:07:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628108/ [5628109] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:12:51 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628109/ [5628110] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-snaps-balanced} time: 00:27:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628110/ [5628111] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:18:20 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628111/ [5628112] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:14:01 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628112/ [5628113] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache} time: 00:08:29 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628113/ [5628115] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:33:56 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628115/ [5628116] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628116/ [5628117] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:04:51 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628117/ [5628118] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:12:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628118/ [5628119] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:11:52 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628119/ [5628120] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:21:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628120/ [5628121] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm_repos} time: 00:03:17 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628121/ [5628122] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:32:38 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628122/ [5628123] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} time: 00:23:23 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628123/ [5628124] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:20:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628124/ [5628125] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/erasure-code} time: 00:39:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628125/ [5628126] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:05 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628126/ [5628127] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 01:17:50 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628127/ [5628128] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:20:54 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628128/ [5628129] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628129/ [5628130] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:31:47 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628130/ [5628131] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/classic msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:12:20 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628131/ [5628132] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench} time: 01:43:56 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628132/ [5628133] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:23 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628133/ [5628134] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:10:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628134/ [5628135] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:11:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628135/ [5628136] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:23 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628136/ [5628137] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:13:15 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628137/ [5628138] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:10:20 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628138/ [5628139] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:11:48 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628139/ [5628141] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:08:10 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628141/ [5628143] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:19:50 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628143/ [5628145] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 01:34:20 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628145/ [5628147] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:10:06 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628147/ [5628149] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:04:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628149/ [5628151] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:04:09 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628151/ [5628153] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:24:56 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628153/ [5628155] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:15:39 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628155/ [5628157] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:37:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628157/ [5628159] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/one workloads/pool-create-delete} time: 00:16:14 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628159/ [5628161] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:19:47 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628161/ [5628163] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:13:09 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628163/ [5628165] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:31:48 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628165/ [5628167] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:19:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628167/ [5628169] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_orch_cli} time: 00:15:24 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628169/ [5628171] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:30:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628171/ [5628173] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:07:58 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628173/ [5628175] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/mgr} time: 00:08:41 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628175/ [5628177] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:17:40 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628177/ [5628179] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:10:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628179/ [5628181] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:14:39 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628181/ [5628183] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:16:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628183/ [5628185] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:19:59 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628185/ [5628187] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:30:33 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628187/ [5628189] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:10:09 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628189/ [5628191] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:04:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628191/ [5628193] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:10:52 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628193/ [5628195] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:14 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628195/ [5628197] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:17:09 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628197/ [5628198] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628198/ [5628200] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:22:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628200/ [5628202] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:26:57 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628202/ [5628206] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:06:17 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628206/ [5628208] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:26:41 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628208/ [5628210] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:51 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628210/ [5628214] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:18:58 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628214/ [5628217] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:29:46 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628217/ [5628218] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} time: 00:20:39 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628218/ [5628219] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:08:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628219/ [5628220] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:13:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628220/ [5628221] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/workunits} time: 00:11:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628221/ [5628222] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 00:30:13 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628222/ [5628223] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:33:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628223/ [5628224] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:21:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628224/ [5628225] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:05:51 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628225/ [5628226] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628226/ [5628227] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/mon_recovery validater/valgrind} time: 00:23:57 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628227/ [5628228] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:11:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628228/ [5628229] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_5925} time: 00:13:11 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628229/ [5628230] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:23:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628230/ [5628231] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:13:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628231/ [5628232] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:56 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628232/ [5628233] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:05:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628233/ [5628234] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:16:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628234/ [5628236] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:06:30 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628236/ [5628237] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:24:26 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628237/ [5628238] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:12:41 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628238/ [5628239] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:27:47 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628239/ [5628240] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/repair_test} time: 00:14:05 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628240/ [5628241] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:16:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628241/ [5628242] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:05:13 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628242/ [5628243] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:31:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628243/ [5628244] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-small-objects-overwrites} time: 00:25:43 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628244/ [5628245] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:46 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628245/ [5628246] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:38:56 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628246/ [5628247] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:11:46 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628247/ [5628248] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache} time: 00:17:39 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628248/ [5628249] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:52 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628249/ [5628250] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:34 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628250/ [5628251] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:29:15 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628251/ [5628252] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:36:17 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628252/ [5628253] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:10:39 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628253/ [5628254] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:25:53 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628254/ [5628255] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:12:19 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628255/ [5628256] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:05:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628256/ [5628257] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:19:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628257/ [5628258] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/mon} time: 00:42:54 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628258/ [5628260] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:34:23 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628260/ [5628261] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:07:32 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628261/ [5628262] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/radosbench} time: 02:15:53 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628262/ [5628263] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:17:01 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628263/ [5628264] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/crash} time: 00:08:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628264/ [5628265] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:28:35 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628265/ [5628266] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:07:17 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628266/ [5628267] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect} time: 00:09:58 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628267/ [5628268] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:10:36 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628268/ [5628269] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:10:01 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628269/ [5628270] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:02 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628270/ [5628271] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/mon_recovery validater/lockdep} time: 00:09:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628271/ [5628273] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:21:09 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628273/ [5628274] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:10:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628274/ [5628275] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:12:10 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628275/ [5628276] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:08 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628276/ [5628277] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:11:25 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628277/ [5628278] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:18:12 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628278/ [5628279] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:14:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628279/ [5628281] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/dashboard} time: 01:07:04 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628281/ [5628282] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:06:40 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628282/ [5628283] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:12:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628283/ [5628284] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:24:45 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628284/ [5628285] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:20:07 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628285/ [5628286] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:21 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628286/ [5628287] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:37:27 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628287/ [5628288] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:26:05 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628288/ [5628289] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:09:28 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628289/ [5628290] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:20:06 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628290/ [5628292] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:04 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628292/ [5628293] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:12:54 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628293/ [5628294] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects} time: 00:21:53 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628294/ [5628295] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:15:18 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628295/ [5628297] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:12:31 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628297/ [5628298] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:15:00 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628298/ [5628299] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:08:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628299/ [5628300] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:35:09 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628300/ [5628301] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:05:44 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628301/ [5628302] rados/objectstore/{backends/objectstore supported-random-distro$/{rhel_8}} time: 05:14:22 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628302/ [5628303] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/test_rbd_api} time: 00:21:10 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628303/ [5628304] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:44:04 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628304/ [5628305] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:21:14 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628305/ [5628306] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/failover} time: 00:15:42 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628306/ [5628307] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:32:46 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628307/ [5628309] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:11:10 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628309/ [5628310] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:05:56 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628310/ [5628311] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:34:13 info: http://pulpito.ceph.com/ksirivad-2020-11-16_04:02:02-rados-wip-mgr-progress-global-efficiency-distro-basic-smithi/5628311/