2020-11-27T02:17:37.790 INFO:root:teuthology version: 1.0.0-e0aaa9bf 2020-11-27T02:17:37.791 INFO:teuthology.results:subset: '9000/10000' 2020-11-27T02:17:37.791 INFO:teuthology.results:seed: '8494' 2020-11-27T06:17:36.179 INFO:root:teuthology version: 1.0.0-e0aaa9bf 2020-11-27T06:17:36.834 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-27T18:14:05.438 INFO:teuthology.results:Tests finished! gathering results... 2020-11-27T18:15:03.123 INFO:scrape:Found 446 jobs 2020-11-27T18:15:18.230 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:18.232 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:18.233 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:18.233 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:18.234 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:18.345 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:18.347 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:18.348 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:18.348 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:18.349 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:18.453 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:18.455 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:18.455 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:18.456 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:18.456 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:18.582 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:18.584 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:18.585 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:18.586 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:18.586 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:18.722 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:18.724 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:18.725 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:18.725 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:18.726 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:18.866 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:18.868 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:18.869 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:18.869 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:18.870 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:18.992 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:18.994 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:18.995 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:18.995 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:18.996 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:19.132 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:19.136 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:19.137 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:19.138 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:19.139 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:19.279 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:19.282 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:19.283 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:19.283 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:19.284 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:19.431 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:19.434 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:19.434 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:19.435 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:19.436 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:19.556 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:19.558 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:19.558 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:19.559 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:19.560 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:19.723 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:19.726 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:19.727 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:19.728 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:19.730 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:19.884 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-7549-ge695ec8e (e695ec8eab7b8e64d8957455b0877d330b05936d) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fa1e709f4d2] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55a8100f3874] 2020-11-27T18:15:19.886 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-11-27T02:40:53.473 INFO:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-11-27T02:40:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2020-11-27T18:15:19.887 WARNING:scrape:Ignoring malparsed backtrace: 2020-11-27T02:42:15.834 INFO:teuthology.orchestra.run:waiting for 300 , 2020-11-27T02:42:15.868 INFO:tasks.ceph.osd.3:Stopped , 2020-11-27T02:42:15.869 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-27T18:15:19.887 WARNING:scrape:Ignoring malparsed backtrace: bash: ntpq: command not found , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 , 2020-11-27T02:44:52.293 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-27T18:15:19.888 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 405, in get_file , self.run(args=args) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run 2020-11-27T18:15:19.910 INFO:scrape:Found 14 distinct failure reasons 2020-11-27T18:15:19.912 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi055 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2020-11-27T18:15:19.912 INFO:scrape:2 jobs: ['5659092', '5659255'] 2020-11-27T18:15:19.913 INFO:scrape:suites intersection: ['task/test_cephadm}'] 2020-11-27T18:15:19.913 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/centos_latest', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2020-11-27T18:15:19.914 INFO:scrape: 2020-11-27T18:15:19.915 INFO:scrape:Failure: Command failed (workunit test cephadm/test_adoption.sh) on smithi170 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' 2020-11-27T18:15:19.915 INFO:scrape:3 jobs: ['5659379', '5659207', '5659052'] 2020-11-27T18:15:19.916 INFO:scrape:suites intersection: ['task/test_adoption}'] 2020-11-27T18:15:19.916 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/centos_latest', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2020-11-27T18:15:19.916 INFO:scrape: 2020-11-27T18:15:19.917 INFO:scrape:Failure: Command failed (workunit test mon/mon-last-epoch-clean.sh) on smithi124 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/mon-last-epoch-clean.sh' 2020-11-27T18:15:19.917 INFO:scrape:2 jobs: ['5659121', '5659343'] 2020-11-27T18:15:19.918 INFO:scrape:suites intersection: ['workloads/mon}'] 2020-11-27T18:15:19.918 INFO:scrape:suites union: ['rados/standalone/{mon_election/classic', 'rados/standalone/{mon_election/connectivity', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/mon}'] 2020-11-27T18:15:19.919 INFO:scrape: 2020-11-27T18:15:19.919 INFO:scrape:Failure: "2020-11-27T05:13:26.900517+0000 osd.10 (osd.10) 82 : cluster [ERR] 5.9 deep-scrub : stat mismatch, got 5/5 objects, 0/0 clones, 0/0 dirty, 0/0 omap, 0/0 pinned, 5/5 hit_set_archive, 0/0 whiteouts, 9488/0 bytes, 0/0 manifest objects, 9488/9488 hit_set_archive bytes." in cluster log 2020-11-27T18:15:19.919 INFO:scrape:1 jobs: ['5658996'] 2020-11-27T18:15:19.920 INFO:scrape:suites: ['1-install/nautilus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-octopus', '8-workload/{rbd-python', 'bluestore-bitmap', 'mon_election/classic', 'rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_latest}'] 2020-11-27T18:15:19.920 INFO:scrape: 2020-11-27T18:15:19.921 INFO:scrape:Failure: '/home/ubuntu/cephtest/archive/syslog/misc.log:2020-11-27T05:40:07.853777+00:00 smithi129 bash[8803]: debug 2020-11-27T05:40:07.847+0000 7f9f41148700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr:INFO:cephadm:Deploying daemon prometheus.a ... ' in syslog 2020-11-27T18:15:19.921 INFO:scrape:1 jobs: ['5659288'] 2020-11-27T18:15:19.922 INFO:scrape:suites: ['2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{ubuntu_18.04}', 'fixed-2', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2020-11-27T18:15:19.922 INFO:scrape: 2020-11-27T18:15:19.923 INFO:scrape:Failure: Command failed on smithi083 with status 5: 'sudo systemctl stop ceph-a72bc886-305c-11eb-980d-001a4aab830c@mon.a' 2020-11-27T18:15:19.923 INFO:scrape:2 jobs: ['5659022', '5659239'] 2020-11-27T18:15:19.923 INFO:scrape:suites intersection: ['2-repo_digest/defaut', '3-start-upgrade', '4-wait', 'fixed-2', 'mon_election/classic}', 'rados/cephadm/upgrade/{1-start'] 2020-11-27T18:15:19.924 INFO:scrape:suites union: ['2-repo_digest/defaut', '3-start-upgrade', '4-wait', 'distro$/{ubuntu_18.04}', 'distro$/{ubuntu_latest}', 'fixed-2', 'mon_election/classic}', 'rados/cephadm/upgrade/{1-start'] 2020-11-27T18:15:19.924 INFO:scrape: 2020-11-27T18:15:19.925 INFO:scrape:Failure: Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) 2020-11-27T18:15:19.925 INFO:scrape:2 jobs: ['5659167', '5659000'] 2020-11-27T18:15:19.926 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_orch_cli}'] 2020-11-27T18:15:19.926 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_orch_cli}'] 2020-11-27T18:15:19.927 INFO:scrape: 2020-11-27T18:15:19.927 INFO:scrape:Failure: reached maximum tries (180) after waiting for 180 seconds 2020-11-27T18:15:19.927 INFO:scrape:2 jobs: ['5659177', '5659122'] 2020-11-27T18:15:19.928 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', 'distro$/{rhel_8.0}', 'fixed-2', 'rados/cephadm/upgrade/{1-start'] 2020-11-27T18:15:19.928 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{rhel_8.0}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2020-11-27T18:15:19.929 INFO:scrape: 2020-11-27T18:15:19.929 INFO:scrape:Dead: b'2020-11-27T18:10:57.796 DEBUG:teuthology.exit:Finished running handlers' 2020-11-27T18:15:19.930 INFO:scrape:1 jobs: ['5659383'] 2020-11-27T18:15:19.930 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zlib', 'rados', 'rados/singleton/{all/thrash-eio', 'supported-random-distro$/{ubuntu_latest}}'] 2020-11-27T18:15:19.933 INFO:scrape: 2020-11-27T18:15:19.933 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi051 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2020-11-27T18:15:19.934 INFO:scrape:2 jobs: ['5659002', '5659222'] 2020-11-27T18:15:19.934 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2020-11-27T18:15:19.935 INFO:scrape:suites union: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2020-11-27T18:15:19.935 INFO:scrape: 2020-11-27T18:15:19.936 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi185 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2020-11-27T18:15:19.936 INFO:scrape:3 jobs: ['5659087', '5659111', '5659242'] 2020-11-27T18:15:19.937 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'msgr-failures/few', 'openstack}', 'rados'] 2020-11-27T18:15:19.937 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/crush-compat', 'd-thrash/default/{default', 'd-thrash/none', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'rados/verify/{centos_latest', 'supported-random-distro$/{rhel_8}', 'tasks/rados_api_tests', 'thrashers/none', 'thrashosds-health', 'thrashosds-health}', 'validater/lockdep}', 'validater/valgrind}', 'workloads/rados_api_tests}'] 2020-11-27T18:15:19.937 INFO:scrape: 2020-11-27T18:15:19.938 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/mon/caps.sh 2020-11-27T18:15:19.938 INFO:scrape:1 jobs: ['5659088'] 2020-11-27T18:15:19.939 INFO:scrape:suites: ['clusters/3-mons', 'mon_election/connectivity', 'msgr-failures/mon-delay', 'msgr/async-v1only', 'objectstore/bluestore-comp-zlib', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/one', 'workloads/rados_mon_workunits}'] 2020-11-27T18:15:19.939 INFO:scrape: 2020-11-27T18:15:19.940 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' 2020-11-27T18:15:19.940 INFO:scrape:1 jobs: ['5658978'] 2020-11-27T18:15:19.940 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/many', 'msgr/async', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'supported-random-distro$/{centos_8}}'] 2020-11-27T18:15:19.941 INFO:scrape: 2020-11-27T18:15:19.941 INFO:scrape:Failure: "2020-11-27T04:18:05.451139+0000 osd.4 (osd.4) 183 : cluster [ERR] scrub 58.0 58:0519addf:test-rados-api-smithi062-30213-74::foo:24 : size 0 != clone_size 10" in cluster log 2020-11-27T18:15:19.942 INFO:scrape:1 jobs: ['5659183'] 2020-11-27T18:15:19.942 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-recovery}', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr/async', 'objectstore/bluestore-comp-zlib', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{rhel_8}', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/rados_api_tests}'] 2020-11-27T18:15:19.943 INFO:scrape: 2020-11-27T18:15:19.962 INFO:teuthology.results:Sending results to bhubbard@redhat.com: Test Run: bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/ failed: 23 dead: 1 running: 0 waiting: 0 queued: 0 passed: 422 Fail ================================================================= [5658978] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:13:10 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658978/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658978/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [5658996] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_latest} ----------------------------------------------------------------- time: 05:01:50 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658996/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658996/ "2020-11-27T05:13:26.900517+0000 osd.10 (osd.10) 82 : cluster [ERR] 5.9 deep-scrub : stat mismatch, got 5/5 objects, 0/0 clones, 0/0 dirty, 0/0 omap, 0/0 pinned, 5/5 hit_set_archive, 0/0 whiteouts, 9488/0 bytes, 0/0 manifest objects, 9488/9488 hit_set_archive bytes." in cluster log [5659000] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_orch_cli} ----------------------------------------------------------------- time: 00:11:08 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659000/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659000/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6251bca41c8f40d094422080a9f46167 Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) [5659002] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:19:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659002/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659002/ sentry: https://sentry.ceph.com/organizations/ceph/?query=922d5900d3614925b9a03be9a8a74d32 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi051 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5659022] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:02:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659022/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659022/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e10f8fef2291429da99093e76b28abdf Command failed on smithi083 with status 5: 'sudo systemctl stop ceph-a72bc886-305c-11eb-980d-001a4aab830c@mon.a' [5659052] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:05:55 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659052/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659052/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c8685e3f6795498ca104c022c16ebbd6 Command failed (workunit test cephadm/test_adoption.sh) on smithi158 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5659087] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:21:37 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659087/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659087/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f5a83710f71747769b5acaafa00faf6e Command failed (workunit test rados/test.sh) on smithi185 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5659088] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} ----------------------------------------------------------------- time: 03:15:53 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659088/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659088/ sentry: https://sentry.ceph.com/organizations/ceph/?query=764bc305a25f488e9116e47d5d182df0 Command failed (workunit test mon/caps.sh) on smithi156 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh' [5659092] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:06:55 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659092/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659092/ sentry: https://sentry.ceph.com/organizations/ceph/?query=db4d1e5e4ce3499eae23c52e260abb30 Command failed (workunit test cephadm/test_cephadm.sh) on smithi055 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5659111] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/lockdep} ----------------------------------------------------------------- time: 00:22:36 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659111/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659111/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6b1981106fff4860af67c93fb8add618 Command failed (workunit test rados/test.sh) on smithi122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5659121] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/mon} ----------------------------------------------------------------- time: 00:17:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659121/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659121/ sentry: https://sentry.ceph.com/organizations/ceph/?query=11339aea9a8b45668dde353744e18ed5 Command failed (workunit test mon/mon-last-epoch-clean.sh) on smithi124 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/mon-last-epoch- clean.sh' [5659122] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:19:00 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659122/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659122/ sentry: https://sentry.ceph.com/organizations/ceph/?query=348f069728de4ea6a2786477920b0b34 reached maximum tries (180) after waiting for 180 seconds [5659167] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_orch_cli} ----------------------------------------------------------------- time: 00:10:43 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659167/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659167/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cbed850a0973489daf59b30d519abe07 Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) [5659177] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:19:21 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659177/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659177/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3f68608daf87414e9fafdcb1060c02c9 reached maximum tries (180) after waiting for 180 seconds [5659183] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:31:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659183/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659183/ "2020-11-27T04:18:05.451139+0000 osd.4 (osd.4) 183 : cluster [ERR] scrub 58.0 58:0519addf:test-rados-api-smithi062-30213-74::foo:24 : size 0 != clone_size 10" in cluster log [5659207] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:06:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659207/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659207/ sentry: https://sentry.ceph.com/organizations/ceph/?query=520e46ffa46f40a68f62172aa4c832db Command failed (workunit test cephadm/test_adoption.sh) on smithi070 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5659222] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:19:30 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659222/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659222/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e1850e28bbf4d87aa96bdba148a2157 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi129 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5659239] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:02:14 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659239/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659239/ sentry: https://sentry.ceph.com/organizations/ceph/?query=31eb2b3663cb4a9daa81cc4979aff6ac Command failed on smithi163 with status 5: 'sudo systemctl stop ceph-e124c10e-306c-11eb-980d-001a4aab830c@mon.a' [5659242] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:47:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659242/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659242/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c20d118dd02c4d9c84e728bc1aeac19f Command failed (workunit test rados/test.sh) on smithi040 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5659255] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:06:05 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659255/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659255/ sentry: https://sentry.ceph.com/organizations/ceph/?query=01d0d54f491047b890dfc7382c787655 Command failed (workunit test cephadm/test_cephadm.sh) on smithi100 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5659288] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 03:09:43 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659288/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659288/ '/home/ubuntu/cephtest/archive/syslog/misc.log:2020-11-27T05:40:07.853777+0 0:00 smithi129 bash[8803]: debug 2020-11-27T05:40:07.847+0000 7f9f41148700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr:INFO:cephadm:Deploying daemon prometheus.a ... ' in syslog [5659343] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/mon} ----------------------------------------------------------------- time: 00:16:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659343/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659343/ sentry: https://sentry.ceph.com/organizations/ceph/?query=678623174f2249afb4f177e7f301427d Command failed (workunit test mon/mon-last-epoch-clean.sh) on smithi070 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/mon-last-epoch- clean.sh' [5659379] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:06:50 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659379/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659379/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0c9462a090ae4cce879844f7906eeb00 Command failed (workunit test cephadm/test_adoption.sh) on smithi170 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e695ec8eab7b8e64d8957455b0877d330b05936d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' Dead ================================================================= [5659383] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659383/ log: http://qa-proxy.ceph.com/teuthology/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659383/ Pass ================================================================= [5658951] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:19:17 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658951/ [5658952] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:04 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658952/ [5658953] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:32:07 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658953/ [5658954] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:06:44 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658954/ [5658955] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} time: 00:37:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658955/ [5658956] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm_repos} time: 00:03:49 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658956/ [5658957] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:15 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658957/ [5658958] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:26:47 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658958/ [5658959] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} time: 00:07:28 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658959/ [5658960] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:31:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658960/ [5658961] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:24:46 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658961/ [5658962] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:36:46 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658962/ [5658963] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-agent-small} time: 00:12:33 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658963/ [5658964] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658964/ [5658965] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:19:33 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658965/ [5658966] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:24:37 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658966/ [5658967] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:25:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658967/ [5658968] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:10:12 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658968/ [5658969] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658969/ [5658970] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:07:39 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658970/ [5658971] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:34:59 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658971/ [5658972] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/prometheus} time: 00:10:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658972/ [5658973] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-pool-snaps} time: 00:26:01 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658973/ [5658974] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:13:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658974/ [5658975] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-snaps-balanced} time: 00:25:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658975/ [5658976] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658976/ [5658977] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados tasks/rados_cls_all validater/valgrind} time: 02:04:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658977/ [5658979] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:10:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658979/ [5658980] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:18:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658980/ [5658981] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:08:36 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658981/ [5658982] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:31:18 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658982/ [5658983] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache} time: 00:12:17 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658983/ [5658984] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:08:37 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658984/ [5658985] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658985/ [5658986] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:12:26 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658986/ [5658987] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:23:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658987/ [5658988] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:09:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658988/ [5658989] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:19:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658989/ [5658990] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} time: 00:24:30 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658990/ [5658991] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:18 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658991/ [5658992] rados/objectstore/{backends/alloc-hint supported-random-distro$/{centos_8}} time: 00:06:51 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658992/ [5658993] rados/rest/{mgr-restful supported-random-distro$/{rhel_8}} time: 00:08:38 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658993/ [5658994] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:15:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658994/ [5658995] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/crush} time: 00:08:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658995/ [5658997] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:18:14 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658997/ [5658998] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658998/ [5658999] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 01:02:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5658999/ [5659001] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/radosbench} time: 01:36:03 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659001/ [5659003] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:27:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659003/ [5659004] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:07:33 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659004/ [5659005] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:26:52 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659005/ [5659006] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:13:51 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659006/ [5659007] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/repair_test} time: 00:14:39 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659007/ [5659008] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} time: 00:10:36 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659008/ [5659009] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659009/ [5659010] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:10:53 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659010/ [5659011] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:13:41 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659011/ [5659012] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:09:08 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659012/ [5659013] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/workunits} time: 00:08:15 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659013/ [5659014] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:11:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659014/ [5659015] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:12:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659015/ [5659016] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:12:27 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659016/ [5659017] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:06:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659017/ [5659018] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:26:00 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659018/ [5659019] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:16:40 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659019/ [5659020] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:28:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659020/ [5659021] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/dashboard} time: 01:07:44 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659021/ [5659023] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:07:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659023/ [5659024] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:18:49 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659024/ [5659025] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:25:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659025/ [5659026] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados tasks/mon_recovery validater/lockdep} time: 00:10:28 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659026/ [5659027] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:32:55 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659027/ [5659028] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:21:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659028/ [5659029] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:34:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659029/ [5659030] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:08:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659030/ [5659031] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:08:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659031/ [5659032] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:19:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659032/ [5659033] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:19:52 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659033/ [5659034] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:11:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659034/ [5659035] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:05:21 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659035/ [5659036] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:30:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659036/ [5659037] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:29:04 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659037/ [5659038] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:21:18 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659038/ [5659039] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:07:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659039/ [5659040] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:08:09 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659040/ [5659041] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:33 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659041/ [5659042] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659042/ [5659043] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:08:46 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659043/ [5659044] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:24:46 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659044/ [5659045] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/dashboard} time: 01:07:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659045/ [5659046] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{rhel_8}} time: 00:11:09 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659046/ [5659047] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:59:36 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659047/ [5659048] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 02:03:38 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659048/ [5659049] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:09:49 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659049/ [5659050] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:05:17 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659050/ [5659051] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:11:04 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659051/ [5659053] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:29:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659053/ [5659054] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:09:01 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659054/ [5659055] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 00:15:26 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659055/ [5659056] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:31:15 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659056/ [5659057] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:08:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659057/ [5659058] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/mgr} time: 00:10:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659058/ [5659059] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:06:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659059/ [5659060] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:15:35 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659060/ [5659061] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:11:40 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659061/ [5659062] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:07:09 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659062/ [5659063] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:19:04 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659063/ [5659064] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:15:29 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659064/ [5659065] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:47:34 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659065/ [5659066] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:27 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659066/ [5659067] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/mon_recovery validater/valgrind} time: 00:26:09 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659067/ [5659068] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:25:52 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659068/ [5659069] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:07:55 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659069/ [5659070] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/connectivity} time: 00:15:15 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659070/ [5659071] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:52 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659071/ [5659072] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:09 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659072/ [5659073] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:22:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659073/ [5659074] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:06:34 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659074/ [5659075] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:03 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659075/ [5659076] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:25:08 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659076/ [5659077] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:05:25 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659077/ [5659078] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:13:59 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659078/ [5659079] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_api_tests} time: 00:22:56 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659079/ [5659080] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache} time: 00:11:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659080/ [5659081] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:26:40 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659081/ [5659082] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:35:05 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659082/ [5659083] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:27:14 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659083/ [5659084] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659084/ [5659085] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:28 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659085/ [5659086] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:21:50 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659086/ [5659089] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:49:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659089/ [5659090] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:22:47 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659090/ [5659091] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:24:59 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659091/ [5659093] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:31:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659093/ [5659094] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:07:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659094/ [5659095] rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8}} time: 00:06:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659095/ [5659096] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:13:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659096/ [5659097] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:22:49 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659097/ [5659098] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/radosbench} time: 02:11:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659098/ [5659099] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:09:53 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659099/ [5659100] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:06:43 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659100/ [5659101] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:18:15 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659101/ [5659102] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect} time: 00:10:39 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659102/ [5659103] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:50:26 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659103/ [5659104] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:07:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659104/ [5659105] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:12:03 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659105/ [5659106] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:12:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659106/ [5659107] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:10:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659107/ [5659108] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:12:34 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659108/ [5659109] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659109/ [5659110] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:25:26 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659110/ [5659112] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:16 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659112/ [5659113] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:29:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659113/ [5659114] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:13:55 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659114/ [5659115] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:04:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659115/ [5659116] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:13:47 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659116/ [5659117] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:20:00 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659117/ [5659118] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:14:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659118/ [5659119] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:25 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659119/ [5659120] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:19:17 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659120/ [5659123] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:11:09 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659123/ [5659124] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:05 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659124/ [5659125] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects} time: 00:20:39 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659125/ [5659126] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:09:17 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659126/ [5659127] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:30:40 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659127/ [5659128] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:27 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659128/ [5659129] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} time: 00:23:27 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659129/ [5659130] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:27:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659130/ [5659131] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm_repos} time: 00:03:46 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659131/ [5659132] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/snaps-few-objects} time: 00:24:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659132/ [5659133] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:20:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659133/ [5659134] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:25:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659134/ [5659135] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:11:17 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659135/ [5659136] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:24:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659136/ [5659137] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:12:02 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659137/ [5659138] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:13:38 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659138/ [5659139] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:10:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659139/ [5659140] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:10:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659140/ [5659141] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:11:47 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659141/ [5659142] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:46 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659142/ [5659143] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} time: 02:31:47 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659143/ [5659144] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:17:58 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659144/ [5659145] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:28:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659145/ [5659146] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:24:53 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659146/ [5659147] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:09:21 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659147/ [5659148] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/admin_socket_objecter_requests} time: 00:42:01 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659148/ [5659149] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/radosbench} time: 01:16:48 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659149/ [5659150] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:07:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659150/ [5659151] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-agent-big} time: 00:35:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659151/ [5659152] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659152/ [5659153] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659153/ [5659154] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:18:49 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659154/ [5659155] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/rados_python} time: 00:14:02 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659155/ [5659156] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:40 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659156/ [5659157] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/osd} time: 03:04:44 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659157/ [5659158] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:47 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659158/ [5659159] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/rados_cls_all validater/valgrind} time: 02:08:30 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659159/ [5659160] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-agent-small} time: 00:14:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659160/ [5659161] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:11:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659161/ [5659162] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:23:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659162/ [5659163] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:11:26 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659163/ [5659164] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:07:18 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659164/ [5659165] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps} time: 00:15:49 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659165/ [5659166] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659166/ [5659168] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:20:05 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659168/ [5659169] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:10:04 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659169/ [5659170] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:18:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659170/ [5659171] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:26:58 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659171/ [5659172] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:21:16 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659172/ [5659173] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/pool-create-delete} time: 00:12:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659173/ [5659174] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:13:34 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659174/ [5659175] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:28:37 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659175/ [5659176] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache} time: 00:12:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659176/ [5659178] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:04 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659178/ [5659179] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} time: 00:18:28 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659179/ [5659180] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:24:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659180/ [5659181] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:31:55 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659181/ [5659182] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:25:53 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659182/ [5659184] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:16 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659184/ [5659185] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/scrub} time: 01:36:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659185/ [5659186] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:07:00 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659186/ [5659187] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:53:44 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659187/ [5659188] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:34:12 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659188/ [5659189] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:15:53 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659189/ [5659190] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:07:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659190/ [5659191] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:26:44 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659191/ [5659192] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:36:41 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659192/ [5659193] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:37 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659193/ [5659194] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:25:45 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659194/ [5659195] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659195/ [5659196] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:13:04 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659196/ [5659197] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect} time: 00:10:27 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659197/ [5659198] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:36 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659198/ [5659199] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/mon_recovery validater/lockdep} time: 00:10:56 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659199/ [5659200] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:02 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659200/ [5659201] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:15:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659201/ [5659202] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:10:01 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659202/ [5659203] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:19:53 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659203/ [5659204] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:07:12 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659204/ [5659205] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:11:14 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659205/ [5659206] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/cosbench_64K_write} time: 00:20:40 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659206/ [5659208] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:20:45 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659208/ [5659209] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} time: 00:17:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659209/ [5659210] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:15:07 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659210/ [5659211] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:20:09 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659211/ [5659212] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:17:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659212/ [5659213] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:12:02 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659213/ [5659214] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:59 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659214/ [5659215] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-localized} time: 00:20:49 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659215/ [5659216] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:43 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659216/ [5659217] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:14:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659217/ [5659218] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/crush} time: 00:08:58 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659218/ [5659219] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_latest} time: 05:15:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659219/ [5659220] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_5925} time: 00:09:45 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659220/ [5659221] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:22:00 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659221/ [5659223] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects} time: 00:24:50 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659223/ [5659224] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:12:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659224/ [5659225] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{centos_8}} time: 00:12:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659225/ [5659226] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:27:33 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659226/ [5659227] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:15:52 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659227/ [5659228] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_striper} time: 00:08:50 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659228/ [5659229] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:14:00 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659229/ [5659230] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} time: 00:22:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659230/ [5659231] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:36 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659231/ [5659232] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:18 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659232/ [5659233] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:27:27 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659233/ [5659234] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:12:28 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659234/ [5659235] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:11:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659235/ [5659236] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:06:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659236/ [5659237] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:05:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659237/ [5659238] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} time: 00:12:35 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659238/ [5659240] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:29:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659240/ [5659241] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:42 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659241/ [5659243] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:47 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659243/ [5659244] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:05:35 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659244/ [5659245] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:13:58 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659245/ [5659246] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:30:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659246/ [5659247] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:05:59 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659247/ [5659248] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-agent-big} time: 00:24:55 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659248/ [5659249] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:31:33 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659249/ [5659250] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:09:47 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659250/ [5659251] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:34:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659251/ [5659252] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-agent-small} time: 00:13:36 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659252/ [5659253] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:11:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659253/ [5659254] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:23:16 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659254/ [5659256] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:06:45 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659256/ [5659257] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} time: 00:29:28 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659257/ [5659258] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:01 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659258/ [5659259] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:20:14 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659259/ [5659260] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_api_tests} time: 00:23:15 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659260/ [5659261] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:08:44 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659261/ [5659262] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:31:12 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659262/ [5659263] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659263/ [5659264] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps-balanced} time: 00:20:37 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659264/ [5659265] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big} time: 00:29:58 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659265/ [5659266] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:16:10 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659266/ [5659267] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:07:55 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659267/ [5659268] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:18:33 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659268/ [5659269] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/prometheus} time: 00:10:02 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659269/ [5659270] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:10:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659270/ [5659271] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:05:38 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659271/ [5659272] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-overwrites} time: 00:21:45 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659272/ [5659273] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache} time: 00:11:56 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659273/ [5659274] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:05:03 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659274/ [5659275] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:22:02 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659275/ [5659276] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:12 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659276/ [5659277] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:25:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659277/ [5659278] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:19:03 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659278/ [5659279] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:09:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659279/ [5659280] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:20:40 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659280/ [5659281] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/mgr} time: 00:10:27 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659281/ [5659282] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:06:14 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659282/ [5659283] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:11:46 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659283/ [5659284] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:46:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659284/ [5659285] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:50 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659285/ [5659286] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/rados_cls_all validater/lockdep} time: 00:15:26 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659286/ [5659287] rados/singleton/{all/radostool mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:14:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659287/ [5659289] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench} time: 01:34:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659289/ [5659290] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:08:17 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659290/ [5659291] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:08:42 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659291/ [5659292] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_latest}} time: 00:11:21 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659292/ [5659293] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:17:03 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659293/ [5659294] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm_repos} time: 00:03:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659294/ [5659295] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:12:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659295/ [5659296] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:08:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659296/ [5659297] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:11:10 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659297/ [5659298] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:08 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659298/ [5659299] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:19:51 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659299/ [5659300] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mix} time: 00:20:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659300/ [5659301] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:12:08 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659301/ [5659302] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:05 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659302/ [5659303] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:48 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659303/ [5659304] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_mon_osdmap_prune} time: 00:55:56 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659304/ [5659305] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:23:10 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659305/ [5659306] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:13:44 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659306/ [5659307] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:38:16 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659307/ [5659308] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:20:07 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659308/ [5659309] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:07:42 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659309/ [5659310] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:30:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659310/ [5659311] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/workunits} time: 00:08:05 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659311/ [5659312] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:13:05 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659312/ [5659313] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:25:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659313/ [5659314] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:19:12 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659314/ [5659315] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:12:56 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659315/ [5659316] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:08:56 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659316/ [5659317] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:04 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659317/ [5659318] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:17:40 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659318/ [5659319] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:08:30 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659319/ [5659320] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:18:02 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659320/ [5659321] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:26:56 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659321/ [5659322] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:06:43 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659322/ [5659323] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:18:49 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659323/ [5659324] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:07:34 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659324/ [5659325] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:12:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659325/ [5659326] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:26:38 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659326/ [5659327] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:10:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659327/ [5659328] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:25 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659328/ [5659329] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/mon_recovery validater/valgrind} time: 00:25:12 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659329/ [5659330] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:23:44 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659330/ [5659331] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:24:35 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659331/ [5659332] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:12:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659332/ [5659333] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:28:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659333/ [5659334] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:05:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659334/ [5659335] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:07:40 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659335/ [5659336] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} time: 00:16:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659336/ [5659337] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:29:34 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659337/ [5659338] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:36:39 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659338/ [5659339] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659339/ [5659340] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:25:26 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659340/ [5659341] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:24:07 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659341/ [5659342] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:29:22 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659342/ [5659344] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:08:45 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659344/ [5659345] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:14:02 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659345/ [5659346] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:11:20 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659346/ [5659347] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/pool-create-delete} time: 00:14:12 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659347/ [5659348] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:06:50 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659348/ [5659349] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:29:57 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659349/ [5659350] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:28 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659350/ [5659351] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:08:38 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659351/ [5659352] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/classic} time: 00:14:34 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659352/ [5659353] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:27:29 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659353/ [5659354] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:08:06 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659354/ [5659355] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:28:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659355/ [5659356] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:20:50 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659356/ [5659357] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:11:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659357/ [5659358] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:11 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659358/ [5659359] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:20:03 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659359/ [5659360] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:10:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659360/ [5659361] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:12:04 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659361/ [5659362] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:14:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659362/ [5659363] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:18:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659363/ [5659364] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:36:17 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659364/ [5659365] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/dashboard} time: 01:10:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659365/ [5659366] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache} time: 00:09:27 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659366/ [5659367] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:05:08 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659367/ [5659368] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:31:24 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659368/ [5659369] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects} time: 00:27:25 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659369/ [5659370] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:58:00 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659370/ [5659371] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:05 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659371/ [5659372] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:26 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659372/ [5659373] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:26 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659373/ [5659374] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_api_tests validater/lockdep} time: 00:22:19 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659374/ [5659375] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:10:21 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659375/ [5659376] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:21:02 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659376/ [5659377] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rados_api_tests} time: 00:24:45 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659377/ [5659378] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/osd} time: 03:03:31 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659378/ [5659380] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency} time: 00:52:36 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659380/ [5659381] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:19:53 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659381/ [5659382] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench} time: 02:12:51 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659382/ [5659384] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:11:39 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659384/ [5659385] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect} time: 00:08:38 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659385/ [5659386] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:15:21 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659386/ [5659387] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:11:21 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659387/ [5659388] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_5925} time: 00:08:32 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659388/ [5659389] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 02:06:00 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659389/ [5659390] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:08:48 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659390/ [5659391] rados/objectstore/{backends/objectstore supported-random-distro$/{rhel_8}} time: 05:01:33 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659391/ [5659392] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/redirect_promote_tests} time: 00:18:13 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659392/ [5659393] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:58 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659393/ [5659394] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:19:54 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659394/ [5659395] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:12:23 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659395/ [5659396] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect_set_object} time: 00:12:52 info: http://pulpito.ceph.com/bhubbard-2020-11-27_02:16:31-rados-wip-badone-testing-distro-basic-smithi/5659396/