2021-01-28T05:16:34.268 INFO:root:teuthology version: 1.0.0-f185165d 2021-01-28T05:16:34.270 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-01-28T16:48:58.845 INFO:teuthology.results:Tests finished! gathering results... 2021-01-28T16:50:38.021 INFO:scrape:Found 354 jobs 2021-01-28T16:50:51.447 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:51.454 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:51.455 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:51.455 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:51.456 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:51.588 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:51.591 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:51.591 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:51.592 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:51.593 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:51.717 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:51.720 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:51.720 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:51.721 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:51.721 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:51.850 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:51.852 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:51.853 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:51.854 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:51.854 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:51.984 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:51.986 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:51.987 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:51.987 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:51.988 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:52.115 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:52.117 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:52.118 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:52.118 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:52.119 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:52.248 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:52.251 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:52.251 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:52.252 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:52.252 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:52.386 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:52.388 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:52.389 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:52.389 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:52.390 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:52.526 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:52.529 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:52.529 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:52.530 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:52.530 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:52.673 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:52.675 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:52.676 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:52.677 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:52.677 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:52.829 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:52.833 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:52.834 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:52.835 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:52.836 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:52.975 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:52.978 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:52.979 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:52.979 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:52.980 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:53.108 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f6d8f99e15a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x55eb603bff26] 2021-01-28T16:50:53.110 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-01-28T04:54:36.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-01-28T04:54:36.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2021-01-28T16:50:53.110 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T04:56:19.792 INFO:tasks.ceph.osd.1:Stopped , 2021-01-28T04:56:19.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit , 2021-01-28T04:56:19.794 INFO:teuthology.orchestra.run:waiting for 300 2021-01-28T16:50:53.111 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:04:42.923 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv01.front.sepia.ceph.com 3 6 377 54 -1761us[-1764us] +/- 104ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 2 6 377 50 +345us[ +342us] +/- 32ms , 2021-01-28T05:04:42.924 INFO:teuthology.orchestra.run.smithi073.stdout:^- hv03.front.sepia.ceph.com 3 6 377 52 +1625us[+1623us] +/- 122ms 2021-01-28T16:50:53.112 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-28T05:05:09.830 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262163 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest , 2021-01-28T05:05:09.844 INFO:teuthology.orchestra.run.smithi073.stdout: 262168 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 28 05:04 /home/ubuntu/cephtest/archive 2021-01-28T16:50:57.807 INFO:scrape:Found 13 distinct failure reasons 2021-01-28T16:50:57.808 INFO:scrape:Failure: reached maximum tries (180) after waiting for 180 seconds 2021-01-28T16:50:57.809 INFO:scrape:5 jobs: ['5831436', '5831304', '5831394', '5831341', '5831488'] 2021-01-28T16:50:57.809 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', 'fixed-2', 'rados/cephadm/upgrade/{1-start'] 2021-01-28T16:50:57.810 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{centos_8.0}', 'distro$/{rhel_8.0}', 'distro$/{rhel_latest}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2021-01-28T16:50:57.810 INFO:scrape: 2021-01-28T16:50:57.811 INFO:scrape:Crash: Test failure: test_turn_off_module (tasks.mgr.test_progress.TestProgress) 2021-01-28T16:50:57.811 INFO:scrape:ceph version 16.1.0-8-g5f17c37f (5f17c37f78a331b7a4bf793890f9d324c64183e5) pacific (rc) 1: /lib64/libpthread.so.0(+0x12b20) [0x7fbf8c720b20] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1b6) [0x55d8a09f37cb] 5: (KernelDevice::_aio_thread()+0x1254) [0x55d8a152abb4] 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x55d8a1535eb1] 7: /lib64/libpthread.so.0(+0x814a) [0x7fbf8c71614a] 8: clone() 2021-01-28T16:50:57.812 INFO:scrape:1 jobs: ['5831423'] 2021-01-28T16:50:57.812 INFO:scrape:suites: ['debug/mgr', 'mon_election/connectivity', 'objectstore/bluestore-stupid', 'rados/mgr/{clusters/{2-node-mgr}', 'supported-random-distro$/{rhel_8}', 'tasks/progress}'] 2021-01-28T16:50:57.813 INFO:scrape: 2021-01-28T16:50:57.813 INFO:scrape:Failure: Command failed on smithi080 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid 4d2e9f18-60d9-11eb-8f9c-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id a --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.80 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' 2021-01-28T16:50:57.814 INFO:scrape:9 jobs: ['5831362', '5831636', '5831447', '5831308', '5831609', '5831619', '5831553', '5831432', '5831364'] 2021-01-28T16:50:57.814 INFO:scrape:suites intersection: [] 2021-01-28T16:50:57.815 INFO:scrape:suites union: ['fixed-2', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr/async', 'msgr/async-v2only', 'rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman', 'rados/cephadm/smoke/{distro/ubuntu_18.04_podman', 'rados/cephadm/with-work/{distro/ubuntu_18.04_podman', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'start', 'start}', 'task/test_orch_cli}', 'tasks/rados_api_tests}', 'tasks/rados_python}'] 2021-01-28T16:50:57.815 INFO:scrape: 2021-01-28T16:50:57.816 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi123 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2021-01-28T16:50:57.816 INFO:scrape:2 jobs: ['5831420', '5831567'] 2021-01-28T16:50:57.817 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/lockdep}'] 2021-01-28T16:50:57.817 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/lockdep}'] 2021-01-28T16:50:57.818 INFO:scrape: 2021-01-28T16:50:57.818 INFO:scrape:Failure: Command failed on smithi163 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'egrep \'"\'"\'(defer backfill|defer recovery)\'"\'"\' /var/log/ceph/ceph-osd.*.log\'' 2021-01-28T16:50:57.819 INFO:scrape:1 jobs: ['5831527'] 2021-01-28T16:50:57.819 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/recovery-preemption', 'supported-random-distro$/{ubuntu_latest}}'] 2021-01-28T16:50:57.820 INFO:scrape: 2021-01-28T16:50:57.820 INFO:scrape:Dead: b'2021-01-28T16:45:49.300 DEBUG:teuthology.exit:Finished running handlers' 2021-01-28T16:50:57.821 INFO:scrape:3 jobs: ['5831618', '5831295', '5831541'] 2021-01-28T16:50:57.821 INFO:scrape:suites intersection: [] 2021-01-28T16:50:57.822 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'fast/normal', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/osd-delay', 'msgr/async', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/objectstore/{backends/objectstore', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}', 'thrashers/minsize_recovery', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-radosbench}', 'workloads/snaps-few-objects-balanced}'] 2021-01-28T16:50:57.822 INFO:scrape: 2021-01-28T16:50:57.823 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi118 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2021-01-28T16:50:57.823 INFO:scrape:3 jobs: ['5831565', '5831417', '5831293'] 2021-01-28T16:50:57.824 INFO:scrape:suites intersection: ['task/test_cephadm}'] 2021-01-28T16:50:57.824 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/centos_latest', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2021-01-28T16:50:57.825 INFO:scrape: 2021-01-28T16:50:57.825 INFO:scrape:Failure: "2021-01-28T05:42:26.405020+0000 osd.10 (osd.10) 50 : cluster [ERR] 5.17 deep-scrub : stat mismatch, got 4/4 objects, 0/0 clones, 0/0 dirty, 0/0 omap, 0/0 pinned, 4/4 hit_set_archive, 0/0 whiteouts, 6781/0 bytes, 0/0 manifest objects, 6781/6781 hit_set_archive bytes." in cluster log 2021-01-28T16:50:57.825 INFO:scrape:1 jobs: ['5831474'] 2021-01-28T16:50:57.826 INFO:scrape:suites: ['1-install/nautilus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-octopus', '8-workload/{rbd-python', 'bluestore-bitmap', 'mon_election/connectivity', 'rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_latest}'] 2021-01-28T16:50:57.827 INFO:scrape: 2021-01-28T16:50:57.827 INFO:scrape:Failure: Found coredumps on ubuntu@smithi150.front.sepia.ceph.com 2021-01-28T16:50:57.827 INFO:scrape:2 jobs: ['5831397', '5831535'] 2021-01-28T16:50:57.828 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2021-01-28T16:50:57.828 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2021-01-28T16:50:57.829 INFO:scrape: 2021-01-28T16:50:57.829 INFO:scrape:Failure: Command failed on smithi111 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid 04d0f89c-6106-11eb-8f9d-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.111:6789]' && sudo chmod +r /etc/ceph/ceph.client.admin.keyring" 2021-01-28T16:50:57.830 INFO:scrape:1 jobs: ['5831452'] 2021-01-28T16:50:57.830 INFO:scrape:suites: ['fixed-2', 'mode/root', 'mon_election/connectivity', 'msgr/async-v1only', 'rados/cephadm/with-work/{distro/ubuntu_18.04_podman', 'start', 'tasks/rados_api_tests}'] 2021-01-28T16:50:57.831 INFO:scrape: 2021-01-28T16:50:57.831 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi114 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2021-01-28T16:50:57.832 INFO:scrape:2 jobs: ['5831476', '5831294'] 2021-01-28T16:50:57.832 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2021-01-28T16:50:57.833 INFO:scrape:suites union: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2021-01-28T16:50:57.834 INFO:scrape: 2021-01-28T16:50:57.834 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' 2021-01-28T16:50:57.835 INFO:scrape:1 jobs: ['5831602'] 2021-01-28T16:50:57.835 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/many', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'supported-random-distro$/{rhel_8}}'] 2021-01-28T16:50:57.836 INFO:scrape: 2021-01-28T16:50:57.836 INFO:scrape:Failure: "2021-01-28T04:30:22.752461+0000 mon.a (mon.0) 145 : cluster [WRN] Health check failed: 71 slow ops, oldest one blocked for 30 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log 2021-01-28T16:50:57.837 INFO:scrape:1 jobs: ['5831561'] 2021-01-28T16:50:57.837 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/many', 'msgr/async-v1only', 'objectstore/bluestore-bitmap', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb', 'supported-random-distro$/{rhel_8}}'] 2021-01-28T16:50:57.838 INFO:scrape: 2021-01-28T16:50:57.851 INFO:teuthology.results:Sending results to ceph-qa@ceph.com: Test Run: teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/ failed: 29 dead: 3 running: 0 waiting: 0 queued: 0 passed: 322 Fail ================================================================= [5831293] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:17:56 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831293/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831293/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4292c4da96264c87a0495be305403778 Command failed (workunit test cephadm/test_cephadm.sh) on smithi165 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5831294] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:37:02 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831294/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831294/ sentry: https://sentry.ceph.com/organizations/ceph/?query=14b5b5b5f6f34d6188861b078a9a3567 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi101 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5831304] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:16:50 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831304/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831304/ sentry: https://sentry.ceph.com/organizations/ceph/?query=76aca0c9918b48c19a21141e5c19075e reached maximum tries (180) after waiting for 180 seconds [5831308] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:07:11 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831308/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831308/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f371e445885b4355825ab8bb5130d8bb Command failed on smithi138 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid a636e9b2-60d4-11eb-8f9c-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.138 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [5831341] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:17:31 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831341/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831341/ sentry: https://sentry.ceph.com/organizations/ceph/?query=80959c6b0cbc43f3a6e323e43a690c86 reached maximum tries (180) after waiting for 180 seconds [5831362] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_orch_cli} ----------------------------------------------------------------- time: 00:05:22 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831362/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831362/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2b0fe1878b414b1daf554c8fe11c180e Command failed on smithi080 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid 4d2e9f18-60d9-11eb-8f9c-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id a --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.80 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [5831364] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:05:02 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831364/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831364/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4855bcf718984b2cab8ff3cf5098dcb1 Command failed on smithi087 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid 645afdb2-60d9-11eb-8f9c-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.87 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [5831394] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:18:53 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831394/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831394/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4700356bd08e4578a7c9a2d2bf5bb70e reached maximum tries (180) after waiting for 180 seconds [5831397] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:17:20 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831397/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831397/ Found coredumps on ubuntu@smithi150.front.sepia.ceph.com [5831417] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:23:23 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831417/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831417/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e3516da7ba2d45cabc0ad2f59e927103 Command failed (workunit test cephadm/test_cephadm.sh) on smithi027 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5831420] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/lockdep} ----------------------------------------------------------------- time: 00:38:34 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831420/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831420/ sentry: https://sentry.ceph.com/organizations/ceph/?query=06ee17cbdad942378754555ea0c95368 Command failed (workunit test rados/test.sh) on smithi123 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5831423] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/progress} ----------------------------------------------------------------- time: 00:31:45 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831423/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831423/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3147531c856c4cc19abe74f174e6705e Test failure: test_turn_off_module (tasks.mgr.test_progress.TestProgress) [5831432] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} ----------------------------------------------------------------- time: 00:04:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831432/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831432/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ea6c6835ef454fda85ffdf159eb52a35 Command failed on smithi176 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid 52b31e92-60e2-11eb-8f9c-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.176 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [5831436] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_latest} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:17:03 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831436/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831436/ sentry: https://sentry.ceph.com/organizations/ceph/?query=63387e20e17047fbaf6f01f708dd95d0 reached maximum tries (180) after waiting for 180 seconds [5831447] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:05:02 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831447/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831447/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b768b976c2a14b2e9585e649c873bb9f Command failed on smithi114 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid 3c6367c8-6105-11eb-8f9d-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.114 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [5831452] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:06:52 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831452/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831452/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d1b92bad391f42e596dd0e283c0db89e Command failed on smithi111 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid 04d0f89c-6106-11eb-8f9d-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial- daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.111:6789]' && sudo chmod +r /etc/ceph/ceph.client.admin.keyring" [5831474] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_latest} ----------------------------------------------------------------- time: 05:30:32 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831474/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831474/ "2021-01-28T05:42:26.405020+0000 osd.10 (osd.10) 50 : cluster [ERR] 5.17 deep-scrub : stat mismatch, got 4/4 objects, 0/0 clones, 0/0 dirty, 0/0 omap, 0/0 pinned, 4/4 hit_set_archive, 0/0 whiteouts, 6781/0 bytes, 0/0 manifest objects, 6781/6781 hit_set_archive bytes." in cluster log [5831476] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:36:29 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831476/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831476/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c0769d5215df4f15942257061ac2aa2d Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi114 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5831488] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:18:05 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831488/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831488/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4678ff2ac0ea4eb78d477ffed2b39862 reached maximum tries (180) after waiting for 180 seconds [5831527] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:19:49 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831527/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831527/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cfb263cd9a2a434095d735606e36e686 Command failed on smithi163 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'egrep \'"\'"\'(defer backfill|defer recovery)\'"\'"\' /var/log/ceph/ceph-osd.*.log\'' [5831535] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:17:49 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831535/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831535/ Found coredumps on ubuntu@smithi118.front.sepia.ceph.com [5831553] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:05:08 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831553/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831553/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e762475d1e284ea1b9a93efb279a23ab Command failed on smithi067 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid bf0c84a2-611e-11eb-8f9e-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.67 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [5831561] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:29:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831561/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831561/ "2021-01-28T04:30:22.752461+0000 mon.a (mon.0) 145 : cluster [WRN] Health check failed: 71 slow ops, oldest one blocked for 30 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log [5831565] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:23:12 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831565/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831565/ sentry: https://sentry.ceph.com/organizations/ceph/?query=81857dbfe14a400ba3a5089da28f2dfc Command failed (workunit test cephadm/test_cephadm.sh) on smithi118 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5831567] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/lockdep} ----------------------------------------------------------------- time: 00:36:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831567/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831567/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9f4f8530b4ba44afb9f52862f11bb27d Command failed (workunit test rados/test.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5f17c37f78a331b7a4bf793890f9d324c64183e5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5831602] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:24:50 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831602/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831602/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [5831609] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:06:56 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831609/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831609/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0c3c5a9d80084ff9a7fa3f07cb9bf6d1 Command failed on smithi168 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid eebeb836-6124-11eb-8f9e-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.168 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [5831619] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} ----------------------------------------------------------------- time: 00:04:21 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831619/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831619/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cbede346a4504f90a7f0fe6ae269551f Command failed on smithi164 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid 8d5e02bc-6125-11eb-8f9e-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.164 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [5831636] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:04:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831636/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831636/ sentry: https://sentry.ceph.com/organizations/ceph/?query=01dc7d4a964a4326a713465cfc7effde Command failed on smithi184 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5f17c37f78a331b7a4bf793890f9d324c64183e5 -v bootstrap --fsid 4c1dc858-6127-11eb-8f9e-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.184 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' Dead ================================================================= [5831295] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831295/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831295/ [5831541] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831541/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831541/ [5831618] rados/objectstore/{backends/objectstore supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831618/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831618/ Pass ================================================================= [5831286] rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_8}} time: 00:24:28 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831286/ [5831287] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:20:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831287/ [5831288] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:51:37 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831288/ [5831289] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:15:57 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831289/ [5831290] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/crush} time: 00:19:52 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831290/ [5831291] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_latest} time: 05:26:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831291/ [5831292] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:21:08 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831292/ [5831296] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{centos_8}} time: 00:13:20 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831296/ [5831297] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:10:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831297/ [5831298] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:21:14 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831298/ [5831299] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:14:21 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831299/ [5831300] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:09 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831300/ [5831301] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831301/ [5831302] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} time: 00:33:45 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831302/ [5831303] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:10:25 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831303/ [5831305] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:07:08 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831305/ [5831306] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/dashboard} time: 01:09:33 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831306/ [5831307] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:23 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831307/ [5831309] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/erasure-code} time: 00:50:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831309/ [5831310] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:18 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831310/ [5831311] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/valgrind} time: 01:01:46 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831311/ [5831312] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:45 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831312/ [5831313] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:20:10 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831313/ [5831314] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:30:34 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831314/ [5831315] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:29:05 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831315/ [5831316] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:59:43 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831316/ [5831317] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:12:57 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831317/ [5831318] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:08:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831318/ [5831319] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read} time: 00:22:23 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831319/ [5831320] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:39 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831320/ [5831321] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_api_tests} time: 00:44:45 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831321/ [5831322] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm_repos} time: 00:17:43 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831322/ [5831323] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:10:15 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831323/ [5831324] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:15 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831324/ [5831325] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:16:05 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831325/ [5831326] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:07 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831326/ [5831327] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{rhel_8}} time: 00:12:59 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831327/ [5831328] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed} time: 00:17:34 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831328/ [5831329] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:09:56 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831329/ [5831330] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:12:31 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831330/ [5831331] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:19:59 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831331/ [5831332] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:42:09 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831332/ [5831333] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:39 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831333/ [5831334] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-overwrites} time: 00:18:13 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831334/ [5831335] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:33:16 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831335/ [5831336] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 01:03:08 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831336/ [5831337] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/mgr} time: 00:24:08 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831337/ [5831338] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:53:07 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831338/ [5831339] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:12:25 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831339/ [5831340] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831340/ [5831342] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/rados_striper} time: 00:09:51 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831342/ [5831343] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 01:02:40 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831343/ [5831344] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:43 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831344/ [5831345] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:20:57 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831345/ [5831346] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:42 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831346/ [5831347] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/lockdep} time: 00:29:51 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831347/ [5831348] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:56 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831348/ [5831349] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-agent-big} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831349/ [5831350] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:12 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831350/ [5831351] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:24 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831351/ [5831352] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:08:27 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831352/ [5831353] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:02 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831353/ [5831354] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:05:03 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831354/ [5831355] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:22:25 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831355/ [5831356] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/insights} time: 00:13:59 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831356/ [5831357] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_osdmap_prune} time: 01:01:21 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831357/ [5831358] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:05:24 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831358/ [5831359] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:43:47 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831359/ [5831360] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-agent-small} time: 00:15:06 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831360/ [5831361] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:22:13 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831361/ [5831363] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/misc} time: 00:40:18 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831363/ [5831365] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:09:10 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831365/ [5831366] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831366/ [5831367] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/dashboard} time: 01:09:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831367/ [5831368] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}} time: 00:09:38 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831368/ [5831369] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:31:45 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831369/ [5831370] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:20:09 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831370/ [5831371] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:24:59 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831371/ [5831372] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:13:14 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831372/ [5831373] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:08:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831373/ [5831374] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big} time: 00:42:25 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831374/ [5831375] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:10:20 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831375/ [5831376] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:11:58 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831376/ [5831377] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:53:16 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831377/ [5831378] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:35:35 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831378/ [5831379] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:10:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831379/ [5831380] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:27:16 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831380/ [5831381] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/radosbench} time: 01:01:44 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831381/ [5831382] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:38 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831382/ [5831383] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/mon_recovery validater/valgrind} time: 00:29:56 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831383/ [5831384] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:03 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831384/ [5831385] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:17:27 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831385/ [5831386] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:01 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831386/ [5831387] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:30:49 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831387/ [5831388] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:13:43 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831388/ [5831389] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:26:01 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831389/ [5831390] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/mon} time: 00:49:59 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831390/ [5831391] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:13:03 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831391/ [5831392] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects} time: 00:23:32 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831392/ [5831393] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/rados_mon_workunits} time: 00:38:57 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831393/ [5831395] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:21:21 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831395/ [5831396] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:19:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831396/ [5831398] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:09:53 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831398/ [5831399] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:18:41 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831399/ [5831400] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:12:53 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831400/ [5831401] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:48 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831401/ [5831402] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:36 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831402/ [5831403] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:35 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831403/ [5831404] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:28:28 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831404/ [5831405] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:44:48 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831405/ [5831406] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:29:38 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831406/ [5831407] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:22:02 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831407/ [5831408] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:29 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831408/ [5831409] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} time: 02:09:41 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831409/ [5831410] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:20:33 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831410/ [5831411] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:19:43 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831411/ [5831412] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:04 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831412/ [5831413] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:20:59 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831413/ [5831414] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/osd} time: 03:25:02 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831414/ [5831415] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:20:37 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831415/ [5831416] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache} time: 00:08:20 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831416/ [5831418] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:09:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831418/ [5831419] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:53 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831419/ [5831421] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:05 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831421/ [5831422] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:37:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831422/ [5831424] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:12:12 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831424/ [5831425] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:18:24 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831425/ [5831426] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:37 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831426/ [5831427] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:28:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831427/ [5831428] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:36:23 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831428/ [5831429] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:09 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831429/ [5831430] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/sync workloads/snaps-few-objects} time: 00:26:30 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831430/ [5831431] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:27:58 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831431/ [5831433] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} time: 00:31:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831433/ [5831434] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:27 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831434/ [5831435] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:19:48 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831435/ [5831437] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:44:07 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831437/ [5831438] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:44:24 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831438/ [5831439] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:29:25 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831439/ [5831440] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/rados_api_tests} time: 00:40:37 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831440/ [5831441] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:21 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831441/ [5831442] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:09:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831442/ [5831443] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/scrub} time: 01:53:42 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831443/ [5831444] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:19:35 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831444/ [5831445] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:19:28 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831445/ [5831446] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:09:08 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831446/ [5831448] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:07 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831448/ [5831449] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:08:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831449/ [5831450] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 01:23:34 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831450/ [5831451] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:34 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831451/ [5831453] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:08 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831453/ [5831454] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/rados_cls_all validater/valgrind} time: 02:15:40 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831454/ [5831455] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:32:25 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831455/ [5831456] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:19:52 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831456/ [5831457] rados/singleton/{all/peer mon_election/connectivity msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:08:43 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831457/ [5831458] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm_repos} time: 00:15:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831458/ [5831459] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/prometheus} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831459/ [5831460] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:33:30 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831460/ [5831461] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/radosbench} time: 01:51:40 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831461/ [5831462] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:05:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831462/ [5831463] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:28:06 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831463/ [5831464] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:08:04 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831464/ [5831465] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/pool-create-delete} time: 00:12:47 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831465/ [5831466] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831466/ [5831467] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:10:01 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831467/ [5831468] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:22:16 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831468/ [5831469] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:23 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831469/ [5831470] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/dashboard} time: 01:12:02 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831470/ [5831471] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:53:12 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831471/ [5831472] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:28 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831472/ [5831473] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/crush} time: 00:23:37 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831473/ [5831475] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:44:15 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831475/ [5831477] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:13:46 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831477/ [5831478] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} time: 00:09:57 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831478/ [5831479] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:23:12 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831479/ [5831480] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:20:29 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831480/ [5831481] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:13:37 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831481/ [5831482] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:13:05 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831482/ [5831483] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:12:38 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831483/ [5831484] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:07:51 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831484/ [5831485] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:05:29 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831485/ [5831486] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/dashboard} time: 01:11:23 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831486/ [5831487] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:19:05 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831487/ [5831489] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:39 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831489/ [5831490] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:19:30 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831490/ [5831491] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:06 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831491/ [5831492] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados tasks/mon_recovery validater/lockdep} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831492/ [5831493] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:54:02 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831493/ [5831496] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:16 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831496/ [5831498] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/workunits} time: 00:20:40 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831498/ [5831499] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:51:27 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831499/ [5831500] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} time: 00:18:50 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831500/ [5831501] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:16:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831501/ [5831502] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:14:25 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831502/ [5831503] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:29:01 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831503/ [5831504] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:13:39 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831504/ [5831505] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:26:34 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831505/ [5831506] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:13:14 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831506/ [5831507] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:17:14 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831507/ [5831508] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_5925} time: 00:09:33 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831508/ [5831509] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:39 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831509/ [5831510] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:16:41 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831510/ [5831511] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:32:15 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831511/ [5831512] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/set-chunks-read} time: 00:18:48 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831512/ [5831513] rados/singleton/{all/random-eio mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:29:10 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831513/ [5831514] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:11:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831514/ [5831515] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:25:22 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831515/ [5831516] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:18:33 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831516/ [5831517] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:14:49 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831517/ [5831518] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:12:36 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831518/ [5831519] rados/singleton/{all/rebuild-mondb mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:16:06 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831519/ [5831520] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:12:44 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831520/ [5831521] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:25:10 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831521/ [5831522] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/mgr} time: 00:22:50 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831522/ [5831523] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:08:53 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831523/ [5831524] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:39:27 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831524/ [5831525] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/connectivity} time: 00:18:51 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831525/ [5831526] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:12:27 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831526/ [5831528] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:43 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831528/ [5831529] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/mon_recovery validater/valgrind} time: 00:30:28 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831529/ [5831530] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/crash} time: 00:12:52 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831530/ [5831531] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:28 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831531/ [5831532] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/dashboard} time: 01:11:08 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831532/ [5831533] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:14:30 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831533/ [5831534] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:22:26 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831534/ [5831536] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects-localized} time: 00:19:31 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831536/ [5831537] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:10:36 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831537/ [5831538] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:48 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831538/ [5831539] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:47:59 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831539/ [5831540] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:48 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831540/ [5831542] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831542/ [5831543] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_api_tests} time: 00:42:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831543/ [5831544] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831544/ [5831545] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831545/ [5831546] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:53 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831546/ [5831547] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects} time: 00:23:02 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831547/ [5831548] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-overwrites} time: 00:23:49 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831548/ [5831549] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:21:23 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831549/ [5831550] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:19:40 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831550/ [5831551] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:37:59 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831551/ [5831552] rados/singleton/{all/test-crash mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:21:10 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831552/ [5831554] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/dashboard} time: 01:09:22 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831554/ [5831555] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:32:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831555/ [5831556] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} time: 00:25:19 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831556/ [5831557] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:20 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831557/ [5831558] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:22 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831558/ [5831559] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:12:21 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831559/ [5831560] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_api_tests} time: 00:38:05 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831560/ [5831562] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:50:12 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831562/ [5831563] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:11:41 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831563/ [5831564] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:10:29 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831564/ [5831566] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:31:52 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831566/ [5831568] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:27:12 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831568/ [5831569] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:12 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831569/ [5831570] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:35 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831570/ [5831571] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:32:34 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831571/ [5831572] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 01:11:24 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831572/ [5831573] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:37 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831573/ [5831574] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:10:44 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831574/ [5831575] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{rhel_8}} time: 00:35:33 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831575/ [5831576] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:13:38 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831576/ [5831577] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:05:16 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831577/ [5831578] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-balanced} time: 00:20:05 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831578/ [5831579] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/mon} time: 00:53:16 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831579/ [5831580] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_mon_osdmap_prune} time: 00:50:55 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831580/ [5831581] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/classic} time: 00:17:55 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831581/ [5831582] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:22:17 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831582/ [5831583] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:28:15 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831583/ [5831584] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} time: 00:15:11 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831584/ [5831585] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:43:28 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831585/ [5831586] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:31:53 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831586/ [5831587] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:28:23 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831587/ [5831588] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:15:31 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831588/ [5831589] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:13:06 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831589/ [5831590] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:15:46 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831590/ [5831591] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:29:18 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831591/ [5831592] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:31:39 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831592/ [5831593] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:08:50 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831593/ [5831594] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:50 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831594/ [5831595] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm_repos} time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831595/ [5831596] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:23:57 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831596/ [5831597] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:17:50 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831597/ [5831598] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:25 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831598/ [5831599] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/insights} time: 00:14:01 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831599/ [5831600] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:30:04 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831600/ [5831601] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} time: 00:48:11 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831601/ [5831603] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/cosbench_64K_write} time: 00:19:45 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831603/ [5831604] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:49 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831604/ [5831605] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/rados_cls_all validater/valgrind} time: 02:20:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831605/ [5831606] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:22 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831606/ [5831607] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:15:48 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831607/ [5831608] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831608/ [5831610] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:18:24 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831610/ [5831611] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:11:45 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831611/ [5831612] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:21:05 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831612/ [5831613] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:14:26 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831613/ [5831614] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:05:40 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831614/ [5831615] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_mon_workunits} time: 00:38:41 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831615/ [5831616] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:30:03 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831616/ [5831617] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:29 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831617/ [5831620] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/connectivity} time: 00:17:38 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831620/ [5831621] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:49 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831621/ [5831622] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:26:00 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831622/ [5831623] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:08:52 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831623/ [5831624] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:16 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831624/ [5831625] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-agent-small} time: 00:13:16 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831625/ [5831626] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831626/ [5831627] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:37:29 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831627/ [5831628] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:52:26 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831628/ [5831629] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_orch_cli} time: 00:19:35 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831629/ [5831630] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:22:35 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831630/ [5831631] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:16:34 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831631/ [5831632] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:08 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831632/ [5831633] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:44:56 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831633/ [5831634] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:25:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831634/ [5831635] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/module_selftest} time: 00:29:32 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831635/ [5831637] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:10 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831637/ [5831638] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/mon_recovery validater/lockdep} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831638/ [5831639] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:13 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831639/ [5831640] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:54 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831640/ [5831641] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:31 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831641/ [5831642] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:06:39 info: http://pulpito.ceph.com/teuthology-2021-01-26_19:05:09-rados-pacific-distro-basic-smithi/5831642/