2020-10-23T13:31:28.206 INFO:root:teuthology version: 1.0.0-f4fe713 2020-10-23T13:31:28.207 INFO:teuthology.results:subset: '916/9999' 2020-10-23T13:31:28.208 INFO:teuthology.results:seed: '2477' 2020-10-23T18:12:17.259 INFO:root:teuthology version: 1.0.0-f4fe713 2020-10-23T18:12:17.384 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-10-24T03:40:40.033 INFO:teuthology.results:Tests finished! gathering results... 2020-10-24T03:41:30.740 INFO:scrape:Found 446 jobs 2020-10-24T03:41:39.115 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6607-g859eb5d2 (859eb5d2de58a5fffc9e4e725f09895d3b5a79c0) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f04e5dd4274] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x27148e) [0x7f04e5dd448e] 2020-10-24T03:41:39.117 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-10-23T15:55:46.677 INFO:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-10-23T15:55:47.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2020-10-24T03:41:39.118 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T15:57:31.995 INFO:teuthology.orchestra.run:waiting for 300 , 2020-10-23T15:57:32.046 INFO:tasks.ceph.osd.5:Stopped , 2020-10-23T15:57:32.047 INFO:teuthology.misc:Shutting down mgr daemons... 2020-10-24T03:41:39.118 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 71ms , 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 12 -288us[ -288us] +/- 58ms , 2020-10-23T16:07:27.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-10-24T03:41:39.119 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status 2020-10-24T03:41:39.243 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6607-g859eb5d2 (859eb5d2de58a5fffc9e4e725f09895d3b5a79c0) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f04e5dd4274] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x27148e) [0x7f04e5dd448e] 2020-10-24T03:41:39.246 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-10-23T15:55:46.677 INFO:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-10-23T15:55:47.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2020-10-24T03:41:39.246 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T15:57:31.995 INFO:teuthology.orchestra.run:waiting for 300 , 2020-10-23T15:57:32.046 INFO:tasks.ceph.osd.5:Stopped , 2020-10-23T15:57:32.047 INFO:teuthology.misc:Shutting down mgr daemons... 2020-10-24T03:41:39.247 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 71ms , 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 12 -288us[ -288us] +/- 58ms , 2020-10-23T16:07:27.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-10-24T03:41:39.247 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status 2020-10-24T03:41:39.376 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6607-g859eb5d2 (859eb5d2de58a5fffc9e4e725f09895d3b5a79c0) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f04e5dd4274] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x27148e) [0x7f04e5dd448e] 2020-10-24T03:41:39.378 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-10-23T15:55:46.677 INFO:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-10-23T15:55:47.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2020-10-24T03:41:39.379 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T15:57:31.995 INFO:teuthology.orchestra.run:waiting for 300 , 2020-10-23T15:57:32.046 INFO:tasks.ceph.osd.5:Stopped , 2020-10-23T15:57:32.047 INFO:teuthology.misc:Shutting down mgr daemons... 2020-10-24T03:41:39.379 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 71ms , 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 12 -288us[ -288us] +/- 58ms , 2020-10-23T16:07:27.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-10-24T03:41:39.380 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status 2020-10-24T03:41:39.507 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6607-g859eb5d2 (859eb5d2de58a5fffc9e4e725f09895d3b5a79c0) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f04e5dd4274] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x27148e) [0x7f04e5dd448e] 2020-10-24T03:41:39.509 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-10-23T15:55:46.677 INFO:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-10-23T15:55:47.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2020-10-24T03:41:39.510 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T15:57:31.995 INFO:teuthology.orchestra.run:waiting for 300 , 2020-10-23T15:57:32.046 INFO:tasks.ceph.osd.5:Stopped , 2020-10-23T15:57:32.047 INFO:teuthology.misc:Shutting down mgr daemons... 2020-10-24T03:41:39.510 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 71ms , 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 12 -288us[ -288us] +/- 58ms , 2020-10-23T16:07:27.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-10-24T03:41:39.511 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status 2020-10-24T03:41:39.638 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6607-g859eb5d2 (859eb5d2de58a5fffc9e4e725f09895d3b5a79c0) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f04e5dd4274] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x27148e) [0x7f04e5dd448e] 2020-10-24T03:41:39.640 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-10-23T15:55:46.677 INFO:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-10-23T15:55:47.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2020-10-24T03:41:39.641 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T15:57:31.995 INFO:teuthology.orchestra.run:waiting for 300 , 2020-10-23T15:57:32.046 INFO:tasks.ceph.osd.5:Stopped , 2020-10-23T15:57:32.047 INFO:teuthology.misc:Shutting down mgr daemons... 2020-10-24T03:41:39.641 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 71ms , 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 12 -288us[ -288us] +/- 58ms , 2020-10-23T16:07:27.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-10-24T03:41:39.641 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status 2020-10-24T03:41:39.768 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6607-g859eb5d2 (859eb5d2de58a5fffc9e4e725f09895d3b5a79c0) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f04e5dd4274] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x27148e) [0x7f04e5dd448e] 2020-10-24T03:41:39.770 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-10-23T15:55:46.677 INFO:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-10-23T15:55:47.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2020-10-24T03:41:39.771 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T15:57:31.995 INFO:teuthology.orchestra.run:waiting for 300 , 2020-10-23T15:57:32.046 INFO:tasks.ceph.osd.5:Stopped , 2020-10-23T15:57:32.047 INFO:teuthology.misc:Shutting down mgr daemons... 2020-10-24T03:41:39.771 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 71ms , 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 12 -288us[ -288us] +/- 58ms , 2020-10-23T16:07:27.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-10-24T03:41:39.772 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status 2020-10-24T03:41:39.904 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6607-g859eb5d2 (859eb5d2de58a5fffc9e4e725f09895d3b5a79c0) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f04e5dd4274] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x27148e) [0x7f04e5dd448e] 2020-10-24T03:41:39.907 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-10-23T15:55:46.677 INFO:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-10-23T15:55:47.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2020-10-24T03:41:39.907 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T15:57:31.995 INFO:teuthology.orchestra.run:waiting for 300 , 2020-10-23T15:57:32.046 INFO:tasks.ceph.osd.5:Stopped , 2020-10-23T15:57:32.047 INFO:teuthology.misc:Shutting down mgr daemons... 2020-10-24T03:41:39.908 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 71ms , 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 12 -288us[ -288us] +/- 58ms , 2020-10-23T16:07:27.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-10-24T03:41:39.909 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status 2020-10-24T03:41:40.035 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6607-g859eb5d2 (859eb5d2de58a5fffc9e4e725f09895d3b5a79c0) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f04e5dd4274] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x27148e) [0x7f04e5dd448e] 2020-10-24T03:41:40.038 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-10-23T15:55:46.677 INFO:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-10-23T15:55:47.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2020-10-24T03:41:40.038 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T15:57:31.995 INFO:teuthology.orchestra.run:waiting for 300 , 2020-10-23T15:57:32.046 INFO:tasks.ceph.osd.5:Stopped , 2020-10-23T15:57:32.047 INFO:teuthology.misc:Shutting down mgr daemons... 2020-10-24T03:41:40.039 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 71ms , 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 12 -288us[ -288us] +/- 58ms , 2020-10-23T16:07:27.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-10-24T03:41:40.039 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status 2020-10-24T03:41:40.172 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-6607-g859eb5d2 (859eb5d2de58a5fffc9e4e725f09895d3b5a79c0) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f04e5dd4274] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x27148e) [0x7f04e5dd448e] 2020-10-24T03:41:40.175 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-10-23T15:55:46.677 INFO:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-10-23T15:55:47.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2020-10-24T03:41:40.175 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T15:57:31.995 INFO:teuthology.orchestra.run:waiting for 300 , 2020-10-23T15:57:32.046 INFO:tasks.ceph.osd.5:Stopped , 2020-10-23T15:57:32.047 INFO:teuthology.misc:Shutting down mgr daemons... 2020-10-24T03:41:40.176 WARNING:scrape:Ignoring malparsed backtrace: 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 71ms , 2020-10-23T16:07:27.000 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 12 -288us[ -288us] +/- 58ms , 2020-10-23T16:07:27.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-10-24T03:41:40.176 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status 2020-10-24T03:42:05.672 INFO:scrape:Found 13 distinct failure reasons 2020-10-24T03:42:05.674 INFO:scrape:Failure: Command failed (workunit test cephtool/test.sh) on smithi104 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh' 2020-10-24T03:42:05.674 INFO:scrape:1 jobs: ['5551341'] 2020-10-24T03:42:05.675 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-bitmap', 'rados', 'rados/singleton-bluestore/{all/cephtool', 'supported-random-distro$/{centos_8}}'] 2020-10-24T03:42:05.675 INFO:scrape: 2020-10-24T03:42:05.675 INFO:scrape:Failure: Command failed on smithi195 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 09ecf5f0-1553-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi195:vg_nvme/lv_4' 2020-10-24T03:42:05.676 INFO:scrape:35 jobs 2020-10-24T03:42:05.676 INFO:scrape:suites intersection: [] 2020-10-24T03:42:05.677 INFO:scrape:suites union: ['1-install/luminous', '1-install/luminous-v1only', '1-install/mimic', '1-install/mimic-v1only', '1-install/nautilus', '1-install/nautilus-v1only', '1-install/nautilus-v2only', '1-install/octopus', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'fixed-2', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v2only', 'rados', 'rados/cephadm/smoke-roleless/{distro/ubuntu_18.04', 'rados/cephadm/smoke-roleless/{distro/ubuntu_latest', 'rados/cephadm/smoke/{distro/ubuntu_18.04', 'rados/cephadm/smoke/{distro/ubuntu_latest', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/cephadm/with-work/{distro/ubuntu_18.04', 'start', 'start}', 'tasks/rados_api_tests}', 'tasks/rados_python}', 'thrashers/careful', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}', 'workloads/radosbench}', 'workloads/rbd_cls}', 'workloads/snaps-few-objects}', 'workloads/test_rbd_api}'] 2020-10-24T03:42:05.677 INFO:scrape: 2020-10-24T03:42:05.678 INFO:scrape:Failure: Command failed (workunit test cephadm/test_adoption.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' 2020-10-24T03:42:05.678 INFO:scrape:3 jobs: ['5551363', '5551025', '5551188'] 2020-10-24T03:42:05.678 INFO:scrape:suites intersection: ['task/test_adoption}'] 2020-10-24T03:42:05.679 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/centos_latest', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2020-10-24T03:42:05.679 INFO:scrape: 2020-10-24T03:42:05.679 INFO:scrape:Failure: Command failed on smithi187 with status 5: 'sudo systemctl stop ceph-772df7f2-1541-11eb-a2ad-001a4aab830c@mon.a' 2020-10-24T03:42:05.680 INFO:scrape:45 jobs 2020-10-24T03:42:05.680 INFO:scrape:suites intersection: [] 2020-10-24T03:42:05.681 INFO:scrape:suites union: ['fixed-2', 'mode/packaged', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'rados/cephadm/dashboard/{distro/centos_latest', 'rados/cephadm/smoke-roleless/{distro/centos_8.0', 'rados/cephadm/smoke-roleless/{distro/centos_latest', 'rados/cephadm/smoke-roleless/{distro/rhel_8.0', 'rados/cephadm/smoke-roleless/{distro/rhel_latest', 'rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman', 'rados/cephadm/smoke/{distro/centos_8.0', 'rados/cephadm/smoke/{distro/centos_latest', 'rados/cephadm/smoke/{distro/rhel_8.0', 'rados/cephadm/smoke/{distro/rhel_latest', 'rados/cephadm/smoke/{distro/ubuntu_18.04_podman', 'rados/cephadm/with-work/{distro/centos_8.0', 'rados/cephadm/with-work/{distro/centos_latest', 'rados/cephadm/with-work/{distro/rhel_8.0', 'rados/cephadm/with-work/{distro/rhel_latest', 'rados/cephadm/with-work/{distro/ubuntu_18.04_podman', 'rados/cephadm/workunits/{distro/centos_latest', 'start', 'start}', 'task/test_e2e}', 'task/test_orch_cli}', 'tasks/rados_api_tests}', 'tasks/rados_python}'] 2020-10-24T03:42:05.681 INFO:scrape: 2020-10-24T03:42:05.681 INFO:scrape:Dead: b'2020-10-24T03:39:36.851 DEBUG:teuthology.exit:Finished running handlers' 2020-10-24T03:42:05.682 INFO:scrape:5 jobs: ['5551248', '5551298', '5551021', '5551140', '5551072'] 2020-10-24T03:42:05.682 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', 'fixed-2', 'rados/cephadm/upgrade/{1-start'] 2020-10-24T03:42:05.682 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{centos_8.0}', 'distro$/{centos_latest}', 'distro$/{rhel_8.0}', 'distro$/{rhel_latest}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2020-10-24T03:42:05.683 INFO:scrape: 2020-10-24T03:42:05.683 INFO:scrape:Failure: "2020-10-23T16:58:14.380051+0000 mon.b (mon.0) 11 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log 2020-10-24T03:42:05.683 INFO:scrape:2 jobs: ['5551308', '5551144'] 2020-10-24T03:42:05.684 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'mon_election/connectivity', 'openstack}', 'rados', 'supported-random-distro$/{centos_8}', 'thrashosds-health'] 2020-10-24T03:42:05.684 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-partial-recovery}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/crush-compat', 'fast/fast', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async-v1only', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/thrash-erasure-code-overwrites/{bluestore-bitmap', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_8}', 'thrashers/careful', 'thrashers/pggrow', 'thrashosds-health', 'workloads/cache-pool-snaps-readproxy}', 'workloads/ec-small-objects-overwrites}'] 2020-10-24T03:42:05.684 INFO:scrape: 2020-10-24T03:42:05.685 INFO:scrape:Failure: "2020-10-23T17:22:56.157383+0000 osd.3 (osd.3) 196 : cluster [ERR] scrub 57.17 57:ea025111:test-rados-api-smithi199-40320-74::foo:24 : size 0 != clone_size 10" in cluster log 2020-10-24T03:42:05.685 INFO:scrape:2 jobs: ['5551332', '5551142'] 2020-10-24T03:42:05.685 INFO:scrape:suites intersection: ['1-pg-log-overrides/short_pg_log', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/crush-compat', 'mon_election/connectivity', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8}', 'thrashosds-health', 'workloads/rados_api_tests}'] 2020-10-24T03:42:05.686 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-partial-recovery}', 'backoff/normal', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/crush-compat', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8}', 'thrashers/default', 'thrashers/pggrow', 'thrashosds-health', 'workloads/rados_api_tests}'] 2020-10-24T03:42:05.686 INFO:scrape: 2020-10-24T03:42:05.686 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' 2020-10-24T03:42:05.687 INFO:scrape:1 jobs: ['5551220'] 2020-10-24T03:42:05.687 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/many', 'msgr/async-v1only', 'objectstore/bluestore-comp-zstd', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'supported-random-distro$/{centos_8}}'] 2020-10-24T03:42:05.687 INFO:scrape: 2020-10-24T03:42:05.688 INFO:scrape:Failure: psutil.NoSuchProcess process no longer exists (pid=12401) 2020-10-24T03:42:05.688 INFO:scrape:2 jobs: ['5551186', '5551414'] 2020-10-24T03:42:05.688 INFO:scrape:suites intersection: ['2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'fixed-2', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2020-10-24T03:42:05.689 INFO:scrape:suites union: ['2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{rhel_8.0}', 'distro$/{ubuntu_18.04_podman}', 'fixed-2', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2020-10-24T03:42:05.689 INFO:scrape: 2020-10-24T03:42:05.689 INFO:scrape:Failure: timed out waiting for admin_socket to appear after osd.0 restart 2020-10-24T03:42:05.689 INFO:scrape:1 jobs: ['5551124'] 2020-10-24T03:42:05.690 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/many', 'msgr/async-v2only', 'objectstore/bluestore-hybrid', 'rados', 'rados/singleton/{all/rebuild-mondb', 'supported-random-distro$/{rhel_8}}'] 2020-10-24T03:42:05.690 INFO:scrape: 2020-10-24T03:42:05.690 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2020-10-24T03:42:05.691 INFO:scrape:3 jobs: ['5551230', '5551400', '5551055'] 2020-10-24T03:42:05.691 INFO:scrape:suites intersection: ['task/test_cephadm}'] 2020-10-24T03:42:05.691 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/centos_latest', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2020-10-24T03:42:05.692 INFO:scrape: 2020-10-24T03:42:05.692 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/mon/caps.sh 2020-10-24T03:42:05.692 INFO:scrape:1 jobs: ['5551109'] 2020-10-24T03:42:05.693 INFO:scrape:suites: ['clusters/3-mons', 'mon_election/connectivity', 'msgr-failures/mon-delay', 'msgr/async-v2only', 'objectstore/bluestore-comp-zlib', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{rhel_8}', 'thrashers/sync-many', 'workloads/rados_mon_workunits}'] 2020-10-24T03:42:05.693 INFO:scrape: 2020-10-24T03:42:05.693 INFO:scrape:Failure: Command failed on smithi133 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo add-apt-repository -y ppa:projectatomic/ppa'" 2020-10-24T03:42:05.694 INFO:scrape:2 jobs: ['5551289', '5551416'] 2020-10-24T03:42:05.694 INFO:scrape:suites intersection: [] 2020-10-24T03:42:05.694 INFO:scrape:suites union: ['clusters/9-mons', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-lz4', 'rados', 'rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman', 'rados/monthrash/{ceph', 'start}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/force-sync-many', 'workloads/rados_5925}'] 2020-10-24T03:42:05.695 INFO:scrape: 2020-10-24T03:42:05.714 INFO:teuthology.results:Sending results to tchaikov@gmail.com: Test Run: kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/ failed: 96 dead: 7 running: 0 waiting: 0 queued: 0 passed: 343 Fail ================================================================= [5551025] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:05:57 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551025/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551025/ sentry: https://sentry.ceph.com/sentry/sepia/?query=26d342df2af5435586c8bdb264bdd621 Command failed (workunit test cephadm/test_adoption.sh) on smithi049 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5551029] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:09:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551029/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551029/ sentry: https://sentry.ceph.com/sentry/sepia/?query=f9a7a0c79ac64d3f9b676048a8373eaa Command failed on smithi051 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb34f2f2-1536-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi051:vg_nvme/lv_4' [5551034] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:03:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551034/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551034/ sentry: https://sentry.ceph.com/sentry/sepia/?query=a64a40c1a5fa447895ab8c7b5fa8f391 Command failed on smithi120 with status 5: 'sudo systemctl stop ceph-15b45472-1536-11eb-a2ad-001a4aab830c@mon.a' [5551037] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} ----------------------------------------------------------------- time: 00:06:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551037/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551037/ sentry: https://sentry.ceph.com/sentry/sepia/?query=ed0d22e29f404857809802ad4d6c9c5f Command failed on smithi081 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca2afa00-1536-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi081:vg_nvme/lv_4' [5551042] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:08:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551042/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551042/ sentry: https://sentry.ceph.com/sentry/sepia/?query=246810da172b48bab33d391cd34dd504 Command failed on smithi164 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46936398-1537-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi164:vg_nvme/lv_4' [5551049] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:06:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551049/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551049/ sentry: https://sentry.ceph.com/sentry/sepia/?query=7c4cb295078a43879d4bd409b589f15a Command failed on smithi145 with status 5: 'sudo systemctl stop ceph-3c661dca-1537-11eb-a2ad-001a4aab830c@mon.a' [5551052] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:07:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551052/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551052/ sentry: https://sentry.ceph.com/sentry/sepia/?query=ce067324ca664708996c745a5ef43e08 Command failed on smithi120 with status 5: 'sudo systemctl stop ceph- eaaf066c-1537-11eb-a2ad-001a4aab830c@mon.a' [5551055] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:06:32 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551055/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551055/ sentry: https://sentry.ceph.com/sentry/sepia/?query=5e151295d6da418c95baddeef0d5d06a Command failed (workunit test cephadm/test_cephadm.sh) on smithi040 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5551059] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:08:47 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551059/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551059/ sentry: https://sentry.ceph.com/sentry/sepia/?query=5228be7fd0a04f07af47fa26af6a6e10 Command failed on smithi139 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7bcb8a6-1539-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi139:vg_nvme/lv_4' [5551062] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:05:56 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551062/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551062/ sentry: https://sentry.ceph.com/sentry/sepia/?query=705d0e5c71b943339942dffb45ceede7 Command failed on smithi001 with status 5: 'sudo systemctl stop ceph-4f457b10-1538-11eb-a2ad-001a4aab830c@mon.a' [5551066] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} ----------------------------------------------------------------- time: 00:04:43 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551066/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551066/ sentry: https://sentry.ceph.com/sentry/sepia/?query=d9b46779af7d4fb6afb6b971c1a24343 Command failed on smithi154 with status 5: 'sudo systemctl stop ceph-0083645a-1539-11eb-a2ad-001a4aab830c@mon.smithi154' [5551077] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:10:34 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551077/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551077/ sentry: https://sentry.ceph.com/sentry/sepia/?query=1719de7757de42b6a20f71d6426511be Command failed on smithi003 with status 5: 'sudo systemctl stop ceph-e6c4d62e-1539-11eb-a2ad-001a4aab830c@mon.a' [5551082] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:06:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551082/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551082/ sentry: https://sentry.ceph.com/sentry/sepia/?query=5b61b1c5b9634e14812db54dca4b3d70 Command failed on smithi006 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be6e02c2-1539-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi006:vg_nvme/lv_4' [5551086] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:08:23 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551086/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551086/ sentry: https://sentry.ceph.com/sentry/sepia/?query=3606b447de124f0c8be4ed256e611342 Command failed on smithi067 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3251f82e-153a-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi067:vg_nvme/lv_4' [5551092] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:05:33 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551092/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551092/ sentry: https://sentry.ceph.com/sentry/sepia/?query=e4425e5bceb24bd89add68f7b5a6b5fb Command failed on smithi204 with status 5: 'sudo systemctl stop ceph-8f5d01ee-153a-11eb-a2ad-001a4aab830c@mon.a' [5551095] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} ----------------------------------------------------------------- time: 00:06:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551095/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551095/ sentry: https://sentry.ceph.com/sentry/sepia/?query=ca5ef1f503e64bf597f50f3fca0165cc Command failed on smithi073 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7d84bbf6-153a-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi073:vg_nvme/lv_4' [5551103] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:08:17 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551103/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551103/ sentry: https://sentry.ceph.com/sentry/sepia/?query=f6c09354ec8d43c28ff2121f1c58aeaf Command failed on smithi157 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bcfb49c0-153b-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi157:vg_nvme/lv_4' [5551106] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} ----------------------------------------------------------------- time: 00:09:22 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551106/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551106/ sentry: https://sentry.ceph.com/sentry/sepia/?query=f8e5aa6724d648f4b6072fbb28af8344 Command failed on smithi199 with status 5: 'sudo systemctl stop ceph-f744985c-153b-11eb-a2ad-001a4aab830c@mon.a' [5551108] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:06:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551108/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551108/ sentry: https://sentry.ceph.com/sentry/sepia/?query=a9042e46a0a844e5a0cab6df79493a49 Command failed on smithi150 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b88dfd2-153b-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi150:vg_nvme/lv_4' [5551109] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_mon_workunits} ----------------------------------------------------------------- time: 03:19:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551109/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551109/ sentry: https://sentry.ceph.com/sentry/sepia/?query=e5b1328bc8eb4a5582d9e8b29543ba27 Command failed (workunit test mon/caps.sh) on smithi086 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh' [5551123] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:08:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551123/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551123/ sentry: https://sentry.ceph.com/sentry/sepia/?query=6903d56595d346b8852de1e669626113 Command failed on smithi143 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f5dcf1a-153d-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi143:vg_nvme/lv_4' [5551124] rados/singleton/{all/rebuild-mondb mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:39:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551124/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551124/ sentry: https://sentry.ceph.com/sentry/sepia/?query=7b9d528924684cd6bc6a746ebf7dc6c0 timed out waiting for admin_socket to appear after osd.0 restart [5551126] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:06:09 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551126/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551126/ sentry: https://sentry.ceph.com/sentry/sepia/?query=4fa40370cd1144408a0076faa5ca58e8 Command failed on smithi089 with status 5: 'sudo systemctl stop ceph-a20b40d8-153c-11eb-a2ad-001a4aab830c@mon.a' [5551134] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:04:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551134/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551134/ sentry: https://sentry.ceph.com/sentry/sepia/?query=c197f7c636894e30b9d081d6101a8c8e Command failed on smithi025 with status 5: 'sudo systemctl stop ceph-e35e0adc-153e-11eb-a2ad-001a4aab830c@mon.a' [5551138] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} ----------------------------------------------------------------- time: 00:04:23 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551138/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551138/ sentry: https://sentry.ceph.com/sentry/sepia/?query=4879d73d28b347bf8fa1e437dafe3f8e Command failed on smithi023 with status 5: 'sudo systemctl stop ceph-d8d596b6-153e-11eb-a2ad-001a4aab830c@mon.smithi023' [5551142] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:26:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551142/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551142/ "2020-10-23T14:59:55.858227+0000 osd.4 (osd.4) 224 : cluster [ERR] scrub 59.13 59:cd66cc1e:test-rados-api-smithi171-37925-74::foo:24 : size 0 != clone_size 10" in cluster log [5551144] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-overwrites} ----------------------------------------------------------------- time: 00:23:30 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551144/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551144/ "2020-10-23T14:54:28.342191+0000 mon.b (mon.0) 11 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log [5551146] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:07:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551146/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551146/ sentry: https://sentry.ceph.com/sentry/sepia/?query=11eb1b0930914a418cab50dfa2b491eb Command failed on smithi168 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34dc9c56-1540-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi168:vg_nvme/lv_4' [5551150] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} ----------------------------------------------------------------- time: 00:06:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551150/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551150/ sentry: https://sentry.ceph.com/sentry/sepia/?query=143d51dc13b1453faaa5762d0386b8c9 Command failed on smithi141 with status 5: 'sudo systemctl stop ceph-c3e70a90-153f-11eb-a2ad-001a4aab830c@mon.a' [5551154] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:03:53 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551154/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551154/ sentry: https://sentry.ceph.com/sentry/sepia/?query=18b2aaaf3c3442b7a4dc7a7c7356c212 Command failed on smithi008 with status 5: 'sudo systemctl stop ceph-431a1028-1540-11eb-a2ad-001a4aab830c@mon.a' [5551160] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:08:34 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551160/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551160/ sentry: https://sentry.ceph.com/sentry/sepia/?query=127f1879eee84eca8c6191edf34fcefe Command failed on smithi085 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 48244aba-1541-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi085:vg_nvme/lv_4' [5551165] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:06:57 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551165/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551165/ sentry: https://sentry.ceph.com/sentry/sepia/?query=88a6e922d7754bbc9f28dfe5f4682855 Command failed on smithi187 with status 5: 'sudo systemctl stop ceph-772df7f2-1541-11eb-a2ad-001a4aab830c@mon.a' [5551170] rados/cephadm/smoke-roleless/{distro/centos_latest start} ----------------------------------------------------------------- time: 00:03:47 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551170/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551170/ sentry: https://sentry.ceph.com/sentry/sepia/?query=c6b4667b7cc84d4690d66f0bc7015840 Command failed on smithi195 with status 5: 'sudo systemctl stop ceph-4f6bc186-1541-11eb-a2ad-001a4aab830c@mon.smithi195' [5551174] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} ----------------------------------------------------------------- time: 00:07:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551174/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551174/ sentry: https://sentry.ceph.com/sentry/sepia/?query=1bc2c81379bc48be979a3dd57f4142c2 Command failed on smithi008 with status 5: 'sudo systemctl stop ceph-8ca5cb22-1542-11eb-a2ad-001a4aab830c@mon.a' [5551177] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:08:57 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551177/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551177/ sentry: https://sentry.ceph.com/sentry/sepia/?query=f400f3552a7042bcb91168e96b841304 Command failed on smithi029 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0fbac96c-1544-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi029:vg_nvme/lv_4' [5551182] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:06:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551182/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551182/ sentry: https://sentry.ceph.com/sentry/sepia/?query=7eb6751ee75d40e08df0deb2f25a1f7e Command failed on smithi025 with status 5: 'sudo systemctl stop ceph- edda2adc-1542-11eb-a2ad-001a4aab830c@mon.a' [5551188] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:06:10 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551188/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551188/ sentry: https://sentry.ceph.com/sentry/sepia/?query=f22897a6ec2c482eb21c1b5f0b0649c2 Command failed (workunit test cephadm/test_adoption.sh) on smithi085 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5551193] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:08:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551193/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551193/ sentry: https://sentry.ceph.com/sentry/sepia/?query=5a5a074a7c064e03ba9b6cf7b225533e Command failed on smithi002 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04f59380-1545-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi002:vg_nvme/lv_4' [5551197] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:06:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551197/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551197/ sentry: https://sentry.ceph.com/sentry/sepia/?query=46e48f43e61d448380aec8da59cea76f Command failed on smithi109 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4cc4db40-1544-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi109:vg_nvme/lv_4' [5551201] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} ----------------------------------------------------------------- time: 00:07:02 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551201/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551201/ sentry: https://sentry.ceph.com/sentry/sepia/?query=67a00bf1b27648b7b93933bc16d3fb53 Command failed on smithi060 with status 5: 'sudo systemctl stop ceph-149272ba-1548-11eb-a2ad-001a4aab830c@mon.smithi060' [5551209] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} ----------------------------------------------------------------- time: 00:06:40 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551209/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551209/ sentry: https://sentry.ceph.com/sentry/sepia/?query=212d78773831422aaa0672b8e6bf0d47 Command failed on smithi117 with status 5: 'sudo systemctl stop ceph-46503fdc-1546-11eb-a2ad-001a4aab830c@mon.a' [5551213] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:05:30 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551213/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551213/ sentry: https://sentry.ceph.com/sentry/sepia/?query=13aece82711e4f72b066c48f4a01456c Command failed on smithi071 with status 5: 'sudo systemctl stop ceph-b1b6b9b8-1546-11eb-a2ad-001a4aab830c@mon.a' [5551218] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:08:27 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551218/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551218/ sentry: https://sentry.ceph.com/sentry/sepia/?query=9a328384fd954378b36bce9eac571bc5 Command failed on smithi070 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d92a10e-1549-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi070:vg_nvme/lv_4' [5551220] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:25:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551220/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551220/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [5551223] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:06:02 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551223/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551223/ sentry: https://sentry.ceph.com/sentry/sepia/?query=bcf273e4e8f9463db96b2554ed486c9e Command failed on smithi085 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 113d2b0a-1548-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi085:vg_nvme/lv_4' [5551226] rados/cephadm/smoke-roleless/{distro/rhel_latest start} ----------------------------------------------------------------- time: 00:05:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551226/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551226/ sentry: https://sentry.ceph.com/sentry/sepia/?query=bd27a965bdce4d76be78909c0bff7f8b Command failed on smithi093 with status 5: 'sudo systemctl stop ceph-6bbb616e-1548-11eb-a2ad-001a4aab830c@mon.smithi093' [5551230] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:06:19 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551230/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551230/ sentry: https://sentry.ceph.com/sentry/sepia/?query=d4396c2fd8654064b7a30d490ab90c64 Command failed (workunit test cephadm/test_cephadm.sh) on smithi122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5551235] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:06:08 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551235/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551235/ sentry: https://sentry.ceph.com/sentry/sepia/?query=f24c807170fb4c6aa4f8dc7e5752804e Command failed on smithi124 with status 5: 'sudo systemctl stop ceph-b3ac903c-1549-11eb-a2ad-001a4aab830c@mon.a' [5551238] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:08:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551238/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551238/ sentry: https://sentry.ceph.com/sentry/sepia/?query=dafc5f8d288241b3af6101a8199c8dd9 Command failed on smithi133 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81c30cb2-154a-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_4' [5551244] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:04:18 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551244/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551244/ sentry: https://sentry.ceph.com/sentry/sepia/?query=eba1b10089bf47ab806a67fa8340158f Command failed on smithi187 with status 5: 'sudo systemctl stop ceph-7695c8e8-154a-11eb-a2ad-001a4aab830c@mon.a' [5551253] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:08:18 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551253/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551253/ sentry: https://sentry.ceph.com/sentry/sepia/?query=73f5c3eab896466fa32c8f2b4ed4aaac Command failed on smithi107 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e594decc-154b-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi107:vg_nvme/lv_4' [5551255] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:03:57 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551255/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551255/ sentry: https://sentry.ceph.com/sentry/sepia/?query=a145fbc88e7c4ca3b52050b994aa9361 Command failed on smithi078 with status 5: 'sudo systemctl stop ceph-b18229e6-154b-11eb-a2ad-001a4aab830c@mon.a' [5551261] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} ----------------------------------------------------------------- time: 00:06:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551261/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551261/ sentry: https://sentry.ceph.com/sentry/sepia/?query=4677566f369e4b4fb49254bf3b728d59 Command failed on smithi096 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid da378502-154b-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi096:vg_nvme/lv_4' [5551267] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:10:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551267/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551267/ sentry: https://sentry.ceph.com/sentry/sepia/?query=c344ed09ccf94839b96ee26bc5f059d1 Command failed on smithi085 with status 5: 'sudo systemctl stop ceph-25ed63c6-154d-11eb-a2ad-001a4aab830c@mon.a' [5551275] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:07:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551275/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551275/ sentry: https://sentry.ceph.com/sentry/sepia/?query=b6a8e0fda99047148f382e7ce3ab44b4 Command failed on smithi099 with status 5: 'sudo systemctl stop ceph- ce881c6a-154d-11eb-a2ad-001a4aab830c@mon.a' [5551278] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:08:49 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551278/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551278/ sentry: https://sentry.ceph.com/sentry/sepia/?query=596c88d0f6c94dae8d7f938100621d91 Command failed on smithi023 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b2cbd236-154e-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi023:vg_nvme/lv_4' [5551283] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:05:46 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551283/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551283/ sentry: https://sentry.ceph.com/sentry/sepia/?query=8ae1cb6d379b489f95c7365bf75d06f9 Command failed on smithi109 with status 5: 'sudo systemctl stop ceph-6c312b5a-154e-11eb-a2ad-001a4aab830c@mon.a' [5551289] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} ----------------------------------------------------------------- time: 00:03:08 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551289/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551289/ sentry: https://sentry.ceph.com/sentry/sepia/?query=39a5b8eda60e4b61bfa90221dcd81883 Command failed on smithi133 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo add-apt-repository -y ppa:projectatomic/ppa'" [5551293] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:09:16 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551293/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551293/ sentry: https://sentry.ceph.com/sentry/sepia/?query=e67169505eba42399acb93ee4368b12e Command failed on smithi079 with status 5: 'sudo systemctl stop ceph-d2bb6b0a-154f-11eb-a2ad-001a4aab830c@mon.a' [5551302] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:08:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551302/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551302/ sentry: https://sentry.ceph.com/sentry/sepia/?query=281b199d0ad442db972faefcdb16b604 Command failed on smithi169 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a689934-1550-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi169:vg_nvme/lv_4' [5551305] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:06:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551305/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551305/ sentry: https://sentry.ceph.com/sentry/sepia/?query=a20ae46003b841a5bfb20f05e7083875 Command failed on smithi071 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66167312-1551-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi071:vg_nvme/lv_4' [5551308] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy} ----------------------------------------------------------------- time: 00:22:22 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551308/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551308/ "2020-10-23T16:58:14.380051+0000 mon.b (mon.0) 11 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log [5551309] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_orch_cli} ----------------------------------------------------------------- time: 00:05:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551309/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551309/ sentry: https://sentry.ceph.com/sentry/sepia/?query=e8f6117ac102420681ac4af191784ed8 Command failed on smithi170 with status 5: 'sudo systemctl stop ceph-8a431bce-1550-11eb-a2ad-001a4aab830c@mon.a' [5551316] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:05:34 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551316/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551316/ sentry: https://sentry.ceph.com/sentry/sepia/?query=0d8125d87494463499f3b4ecbdb8f1c1 Command failed on smithi168 with status 5: 'sudo systemctl stop ceph-5b2e1580-1552-11eb-a2ad-001a4aab830c@mon.a' [5551318] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} ----------------------------------------------------------------- time: 00:06:12 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551318/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551318/ sentry: https://sentry.ceph.com/sentry/sepia/?query=a08b7c71bffa41edb676f17066532869 Command failed on smithi002 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0a9fda8-1551-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi002:vg_nvme/lv_4' [5551321] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:08:43 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551321/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551321/ sentry: https://sentry.ceph.com/sentry/sepia/?query=45adc27dc8f346d8b5a2dc028d73f65a Command failed on smithi006 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0a5b3dc-1552-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi006:vg_nvme/lv_4' [5551325] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} ----------------------------------------------------------------- time: 00:08:02 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551325/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551325/ sentry: https://sentry.ceph.com/sentry/sepia/?query=ad730a4591514cbdbfd976a50c7897ab Command failed on smithi016 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b84626d6-1552-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi016:vg_nvme/lv_4' [5551331] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:06:09 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551331/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551331/ sentry: https://sentry.ceph.com/sentry/sepia/?query=1d89ad901604463b8800b4157000a531 Command failed on smithi195 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 09ecf5f0-1553-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi195:vg_nvme/lv_4' [5551332] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:24:22 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551332/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551332/ "2020-10-23T17:22:56.157383+0000 osd.3 (osd.3) 196 : cluster [ERR] scrub 57.17 57:ea025111:test-rados-api-smithi199-40320-74::foo:24 : size 0 != clone_size 10" in cluster log [5551334] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:08:47 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551334/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551334/ sentry: https://sentry.ceph.com/sentry/sepia/?query=a3edd637caec4e489b5163620e36ae43 Command failed on smithi008 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b25b6190-1553-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi008:vg_nvme/lv_4' [5551341] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:23:20 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551341/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551341/ sentry: https://sentry.ceph.com/sentry/sepia/?query=ebfda0fea0904177a36aeda012cde4ec Command failed (workunit test cephtool/test.sh) on smithi104 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh' [5551346] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:06:11 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551346/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551346/ sentry: https://sentry.ceph.com/sentry/sepia/?query=b441b161789f46b898a271ccd72bd2c2 Command failed on smithi002 with status 5: 'sudo systemctl stop ceph-c3047fae-1553-11eb-a2ad-001a4aab830c@mon.a' [5551352] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:04:19 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551352/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551352/ sentry: https://sentry.ceph.com/sentry/sepia/?query=fd92c702fcb4402fbb1a99831afa6019 Command failed on smithi117 with status 5: 'sudo systemctl stop ceph-7e312cb4-1554-11eb-a2ad-001a4aab830c@mon.a' [5551356] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} ----------------------------------------------------------------- time: 00:04:22 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551356/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551356/ sentry: https://sentry.ceph.com/sentry/sepia/?query=c064fedf4fa441e5afff51afae12f1f9 Command failed on smithi170 with status 5: 'sudo systemctl stop ceph-a1437c84-1554-11eb-a2ad-001a4aab830c@mon.smithi170' [5551363] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:06:21 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551363/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551363/ sentry: https://sentry.ceph.com/sentry/sepia/?query=0dbcac6292ee4176ac6f4d3f428bb8ed Command failed (workunit test cephadm/test_adoption.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5551368] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:07:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551368/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551368/ sentry: https://sentry.ceph.com/sentry/sepia/?query=69572d68a1d143b98363407dee3d9711 Command failed on smithi115 with status 5: 'sudo systemctl stop ceph- cd9586d6-1556-11eb-a2ad-001a4aab830c@mon.a' [5551375] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:10:39 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551375/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551375/ sentry: https://sentry.ceph.com/sentry/sepia/?query=08127d6e6bac4e12b77824b4d42b549e Command failed on smithi112 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39e249e6-1557-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi112:vg_nvme/lv_4' [5551378] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:03:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551378/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551378/ sentry: https://sentry.ceph.com/sentry/sepia/?query=0918f58360224a0ca11c4283a3645066 Command failed on smithi093 with status 5: 'sudo systemctl stop ceph-3fc84c12-1556-11eb-a2ad-001a4aab830c@mon.a' [5551383] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:07:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551383/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551383/ sentry: https://sentry.ceph.com/sentry/sepia/?query=5d8310624e6b4586b7ff33f30ff935a8 Command failed on smithi099 with status 5: 'sudo systemctl stop ceph-114840e4-1557-11eb-a2ad-001a4aab830c@mon.a' [5551387] rados/cephadm/smoke-roleless/{distro/centos_latest start} ----------------------------------------------------------------- time: 00:03:53 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551387/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551387/ sentry: https://sentry.ceph.com/sentry/sepia/?query=5ad5b8bfc31a4e4d968347a89f931e1e Command failed on smithi002 with status 5: 'sudo systemctl stop ceph- beb1c99a-1556-11eb-a2ad-001a4aab830c@mon.smithi002' [5551390] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:08:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551390/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551390/ sentry: https://sentry.ceph.com/sentry/sepia/?query=c3776b3310d546079c806cbf26fb3660 Command failed on smithi065 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bdc95a1a-1557-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi065:vg_nvme/lv_4' [5551395] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:06:24 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551395/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551395/ sentry: https://sentry.ceph.com/sentry/sepia/?query=e5f8166d161b4fc4892be907cda9d72b Command failed on smithi082 with status 5: 'sudo systemctl stop ceph- ccfafb74-1557-11eb-a2ad-001a4aab830c@mon.a' [5551400] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:06:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551400/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551400/ sentry: https://sentry.ceph.com/sentry/sepia/?query=df4c2ee2ed114736bc47c8322a8be458 Command failed (workunit test cephadm/test_cephadm.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5551403] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:05:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551403/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551403/ sentry: https://sentry.ceph.com/sentry/sepia/?query=f09695f0fe4d4ca1a31a80cc6bfc74ee Command failed on smithi106 with status 5: 'sudo systemctl stop ceph-4db88dda-1558-11eb-a2ad-001a4aab830c@mon.a' [5551407] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:08:24 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551407/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551407/ sentry: https://sentry.ceph.com/sentry/sepia/?query=3d0317508bc44bbeb9f5db7bda95be45 Command failed on smithi059 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ac950ba-1559-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi059:vg_nvme/lv_4' [5551416] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_5925} ----------------------------------------------------------------- time: 00:10:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551416/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551416/ sentry: https://sentry.ceph.com/sentry/sepia/?query=25d002730ba244ee97709a9bbd9ede8d Command failed on smithi166 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_rados_delete_pools_parallel --debug_objecter 20 --debug_ms 1 --debug_rados 20 --debug_monc 20'" [5551420] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:06:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551420/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551420/ sentry: https://sentry.ceph.com/sentry/sepia/?query=8620467e73094698b30012d25b66612f Command failed on smithi013 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e13d45b8-1559-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi013:vg_nvme/lv_4' [5551424] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} ----------------------------------------------------------------- time: 00:07:12 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551424/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551424/ sentry: https://sentry.ceph.com/sentry/sepia/?query=e59b838f54074a229cb042e4992585e8 Command failed on smithi065 with status 5: 'sudo systemctl stop ceph-08f7164c-155a-11eb-a2ad-001a4aab830c@mon.smithi065' [5551428] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:06:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551428/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551428/ sentry: https://sentry.ceph.com/sentry/sepia/?query=d839ec72e98240dfb370c1f4e9e6c9b5 Command failed on smithi135 with status 5: 'sudo systemctl stop ceph-2f2e2012-155a-11eb-a2ad-001a4aab830c@mon.a' [5551432] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:08:10 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551432/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551432/ sentry: https://sentry.ceph.com/sentry/sepia/?query=82a95a684b4344c08f24ead88c4553dc Command failed on smithi191 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 72891852-155b-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi191:vg_nvme/lv_4' [5551436] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:05:37 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551436/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551436/ sentry: https://sentry.ceph.com/sentry/sepia/?query=657f5bed186a46539d71e0be96bb7570 Command failed on smithi180 with status 5: 'sudo systemctl stop ceph-36c4fc14-155b-11eb-a2ad-001a4aab830c@mon.a' [5551443] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} ----------------------------------------------------------------- time: 00:10:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551443/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551443/ sentry: https://sentry.ceph.com/sentry/sepia/?query=d2f2cd7e6f094c6da2fae4a6d880daaa Command failed on smithi165 with status 5: 'sudo systemctl stop ceph-f4a99708-155b-11eb-a2ad-001a4aab830c@mon.a' [5551447] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:06:09 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551447/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551447/ sentry: https://sentry.ceph.com/sentry/sepia/?query=bd9cfbe46dd24e6796ebefc58900a5ab Command failed on smithi023 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce2c41e8-155b-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi023:vg_nvme/lv_4' [5551451] rados/cephadm/smoke-roleless/{distro/rhel_latest start} ----------------------------------------------------------------- time: 00:06:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551451/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551451/ sentry: https://sentry.ceph.com/sentry/sepia/?query=480d8c52ec4444b79d587118b9c510b8 Command failed on smithi193 with status 5: 'sudo systemctl stop ceph-e2292da0-155b-11eb-a2ad-001a4aab830c@mon.smithi193' [5551457] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:08:12 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551457/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551457/ sentry: https://sentry.ceph.com/sentry/sepia/?query=c3bacdaabe4e442a90582632891475b8 Command failed on smithi156 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:859eb5d2de58a5fffc9e4e725f09895d3b5a79c0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid acb4f2a2-155c-11eb-a2ad-001a4aab830c -- ceph orch daemon add osd smithi156:vg_nvme/lv_4' [5551463] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:04:25 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551463/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551463/ sentry: https://sentry.ceph.com/sentry/sepia/?query=0dff4b5b20104e1ebec1c304acd2bc83 Command failed on smithi160 with status 5: 'sudo systemctl stop ceph-8cee1a66-155c-11eb-a2ad-001a4aab830c@mon.a' Dead ================================================================= [5551021] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551021/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551021/ [5551072] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551072/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551072/ [5551140] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551140/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551140/ [5551186] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04_podman} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 11:34:24 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551186/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551186/ sentry: https://sentry.ceph.com/sentry/sepia/?query=617b99fa68ac4166b0584b9fb8643b50 psutil.NoSuchProcess process no longer exists (pid=12401) [5551248] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551248/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551248/ [5551298] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551298/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551298/ [5551414] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 09:47:26 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551414/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551414/ sentry: https://sentry.ceph.com/sentry/sepia/?query=4e1c7352fc24467cbab4f33b67116fca psutil.NoSuchProcess process no longer exists (pid=16551) Pass ================================================================= [5551022] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/mon} time: 00:43:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551022/ [5551023] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:22:48 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551023/ [5551024] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551024/ [5551026] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} time: 00:26:30 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551026/ [5551027] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_api_tests} time: 00:28:36 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551027/ [5551028] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:28:47 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551028/ [5551030] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-snaps-balanced} time: 00:18:39 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551030/ [5551031] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:37 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551031/ [5551032] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados tasks/mon_recovery validater/lockdep} time: 00:10:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551032/ [5551033] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:19:23 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551033/ [5551035] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:23:58 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551035/ [5551036] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551036/ [5551038] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache} time: 00:11:15 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551038/ [5551039] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:14:51 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551039/ [5551040] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 00:58:40 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551040/ [5551041] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:40 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551041/ [5551043] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:31:50 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551043/ [5551044] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:15:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551044/ [5551045] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:10:31 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551045/ [5551046] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/crash} time: 00:09:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551046/ [5551047] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:11:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551047/ [5551048] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:22:49 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551048/ [5551050] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:19:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551050/ [5551051] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 01:09:16 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551051/ [5551053] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:08:31 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551053/ [5551054] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/radosbench} time: 01:45:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551054/ [5551056] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:05:47 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551056/ [5551057] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:16:41 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551057/ [5551058] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:32 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551058/ [5551060] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:12:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551060/ [5551061] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:19:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551061/ [5551063] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:02:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551063/ [5551064] rados/objectstore/{backends/objectstore supported-random-distro$/{ubuntu_latest}} time: 05:07:15 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551064/ [5551065] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:17:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551065/ [5551067] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:12:27 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551067/ [5551068] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_osdmap_prune} time: 00:51:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551068/ [5551069] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:27:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551069/ [5551070] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:08:53 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551070/ [5551071] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:13:12 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551071/ [5551073] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:17 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551073/ [5551074] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados tasks/rados_api_tests validater/valgrind} time: 00:43:58 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551074/ [5551075] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:22:08 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551075/ [5551076] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:10:17 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551076/ [5551078] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} time: 00:12:24 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551078/ [5551079] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:17:32 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551079/ [5551080] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:26:27 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551080/ [5551081] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:06:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551081/ [5551083] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551083/ [5551084] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:18:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551084/ [5551085] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:31:19 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551085/ [5551087] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/failover} time: 00:14:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551087/ [5551088] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:32:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551088/ [5551089] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:27:36 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551089/ [5551090] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:09:53 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551090/ [5551091] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 00:58:14 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551091/ [5551093] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:29:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551093/ [5551094] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:11:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551094/ [5551096] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:28:51 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551096/ [5551097] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:07:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551097/ [5551098] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm_repos} time: 00:04:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551098/ [5551099] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:12 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551099/ [5551100] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:10:42 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551100/ [5551101] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:09 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551101/ [5551102] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:20:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551102/ [5551104] rados/singleton/{all/random-eio mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:27:53 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551104/ [5551105] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:17:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551105/ [5551107] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:11:10 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551107/ [5551110] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:27:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551110/ [5551111] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:16:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551111/ [5551112] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 00:59:10 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551112/ [5551113] rados/objectstore/{backends/alloc-hint supported-random-distro$/{ubuntu_latest}} time: 00:06:30 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551113/ [5551114] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:06:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551114/ [5551115] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:32:20 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551115/ [5551116] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:23 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551116/ [5551117] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/crush} time: 00:09:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551117/ [5551118] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_latest} time: 05:07:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551118/ [5551119] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:17:11 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551119/ [5551120] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:21:36 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551120/ [5551121] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551121/ [5551122] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/rados_cls_all validater/lockdep} time: 00:16:50 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551122/ [5551125] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:25:47 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551125/ [5551127] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:25 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551127/ [5551128] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:23:39 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551128/ [5551129] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:20:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551129/ [5551130] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} time: 00:09:20 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551130/ [5551131] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:31 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551131/ [5551132] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:19:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551132/ [5551133] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} tasks/insights} time: 00:13:49 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551133/ [5551135] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:17:57 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551135/ [5551136] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:15:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551136/ [5551137] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:09:11 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551137/ [5551139] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects} time: 00:32:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551139/ [5551141] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:02:53 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551141/ [5551143] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:10:31 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551143/ [5551145] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:09:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551145/ [5551147] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency} time: 01:10:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551147/ [5551148] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/erasure-code} time: 00:39:25 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551148/ [5551149] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:40 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551149/ [5551151] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/radosbench} time: 02:30:23 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551151/ [5551152] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/sample_fio} time: 00:11:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551152/ [5551153] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:06:51 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551153/ [5551155] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect} time: 00:08:41 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551155/ [5551156] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:07:34 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551156/ [5551157] rados/singleton/{all/test-crash mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:07:25 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551157/ [5551158] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/sync workloads/snaps-few-objects} time: 00:28:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551158/ [5551159] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 02:16:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551159/ [5551161] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect_promote_tests} time: 00:13:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551161/ [5551162] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:24:31 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551162/ [5551163] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/mon_recovery validater/valgrind} time: 00:25:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551163/ [5551164] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect_set_object} time: 00:10:57 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551164/ [5551166] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 00:58:37 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551166/ [5551167] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{rhel_8}} time: 00:13:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551167/ [5551168] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:15 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551168/ [5551169] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/set-chunks-read} time: 00:14:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551169/ [5551171] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:08:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551171/ [5551172] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:17:13 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551172/ [5551173] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/small-objects-balanced} time: 00:19:19 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551173/ [5551175] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:24:39 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551175/ [5551176] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} time: 00:22:32 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551176/ [5551178] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:12:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551178/ [5551179] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:46 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551179/ [5551180] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects} time: 00:21:15 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551180/ [5551181] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:08:53 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551181/ [5551183] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:06:34 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551183/ [5551184] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 01:22:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551184/ [5551185] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:25:41 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551185/ [5551187] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} time: 00:30:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551187/ [5551189] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:28:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551189/ [5551190] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:02:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551190/ [5551191] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:27:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551191/ [5551192] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:21:10 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551192/ [5551194] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:32:02 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551194/ [5551195] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:06:47 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551195/ [5551196] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed} time: 00:15:08 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551196/ [5551198] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/pool-create-delete} time: 00:14:57 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551198/ [5551199] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:35:13 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551199/ [5551200] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:21:46 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551200/ [5551202] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-agent-big} time: 00:20:23 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551202/ [5551203] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} time: 00:17:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551203/ [5551204] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:41 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551204/ [5551205] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/rados_api_tests validater/lockdep} time: 00:21:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551205/ [5551206] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:17:36 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551206/ [5551207] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:18 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551207/ [5551208] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:27:22 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551208/ [5551210] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:11:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551210/ [5551211] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-small} time: 00:13:38 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551211/ [5551212] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:22:53 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551212/ [5551214] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:34:17 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551214/ [5551215] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 00:58:26 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551215/ [5551216] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}} time: 00:10:20 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551216/ [5551217] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/progress} time: 00:10:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551217/ [5551219] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:24:37 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551219/ [5551221] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:10:14 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551221/ [5551222] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:20:18 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551222/ [5551224] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:09:38 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551224/ [5551225] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:22:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551225/ [5551227] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:18:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551227/ [5551228] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:29:24 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551228/ [5551229] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache} time: 00:08:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551229/ [5551231] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/cosbench_64K_write} time: 00:21:49 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551231/ [5551232] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:07:20 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551232/ [5551233] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:28:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551233/ [5551234] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551234/ [5551236] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:58 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551236/ [5551237] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/rados_api_tests} time: 00:32:33 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551237/ [5551239] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:04:13 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551239/ [5551240] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 01:07:19 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551240/ [5551241] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_5925} time: 00:11:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551241/ [5551242] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:19:39 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551242/ [5551243] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/mon} time: 00:42:12 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551243/ [5551245] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:06:25 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551245/ [5551246] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/radosbench} time: 02:19:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551246/ [5551247] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:08:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551247/ [5551249] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:51 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551249/ [5551250] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:23:51 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551250/ [5551251] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados tasks/rados_cls_all validater/valgrind} time: 02:12:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551251/ [5551252] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:16:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551252/ [5551254] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:14:26 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551254/ [5551256] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:07:21 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551256/ [5551257] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/prometheus} time: 00:09:29 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551257/ [5551258] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:11:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551258/ [5551259] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:10:34 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551259/ [5551260] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:14:25 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551260/ [5551262] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/set-chunks-read} time: 00:18:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551262/ [5551263] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:29:37 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551263/ [5551264] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:02:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551264/ [5551265] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} time: 02:33:24 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551265/ [5551266] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:18:14 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551266/ [5551268] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:06:43 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551268/ [5551269] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:19:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551269/ [5551270] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:06:38 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551270/ [5551271] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm_repos} time: 00:04:11 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551271/ [5551272] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:25:50 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551272/ [5551273] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/small-objects-localized} time: 00:17:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551273/ [5551274] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:20:21 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551274/ [5551276] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects} time: 00:23:16 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551276/ [5551277] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/osd} time: 03:03:30 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551277/ [5551279] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} time: 00:22:49 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551279/ [5551280] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551280/ [5551281] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:07:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551281/ [5551282] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:29:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551282/ [5551284] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551284/ [5551285] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_api_tests} time: 00:28:02 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551285/ [5551286] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:23:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551286/ [5551287] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551287/ [5551288] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:26:57 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551288/ [5551290] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 00:58:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551290/ [5551291] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551291/ [5551292] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:12:18 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551292/ [5551294] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:41 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551294/ [5551295] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/rados_cls_all validater/lockdep} time: 00:15:40 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551295/ [5551296] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:08:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551296/ [5551297] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} time: 00:31:58 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551297/ [5551299] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:18 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551299/ [5551300] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:21:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551300/ [5551301] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/workunits} time: 00:07:23 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551301/ [5551303] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:09:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551303/ [5551304] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-agent-small} time: 00:13:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551304/ [5551306] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:29:51 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551306/ [5551307] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:46:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551307/ [5551310] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:08:27 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551310/ [5551311] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:08:38 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551311/ [5551312] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps} time: 00:22:57 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551312/ [5551313] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:30:43 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551313/ [5551314] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:02:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551314/ [5551315] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{centos_8}} time: 02:16:30 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551315/ [5551317] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-snaps-balanced} time: 00:24:50 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551317/ [5551319] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:05:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551319/ [5551320] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:22:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551320/ [5551322] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:39 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551322/ [5551323] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache} time: 00:12:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551323/ [5551324] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:11:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551324/ [5551326] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:21:30 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551326/ [5551327] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_mon_osdmap_prune} time: 00:46:39 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551327/ [5551328] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:18:37 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551328/ [5551329] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:08:42 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551329/ [5551330] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:45:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551330/ [5551333] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:24:11 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551333/ [5551335] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:33:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551335/ [5551336] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados tasks/mon_recovery validater/valgrind} time: 00:25:58 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551336/ [5551337] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 00:50:50 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551337/ [5551338] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:13:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551338/ [5551339] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 00:59:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551339/ [5551340] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:11:42 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551340/ [5551342] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:19 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551342/ [5551343] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/crush} time: 00:10:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551343/ [5551344] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_latest} time: 05:11:14 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551344/ [5551345] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench} time: 02:16:37 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551345/ [5551347] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:08:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551347/ [5551348] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:43:49 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551348/ [5551349] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:14:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551349/ [5551350] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} time: 00:09:18 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551350/ [5551351] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:11:56 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551351/ [5551353] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:58 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551353/ [5551354] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:09:41 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551354/ [5551355] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:09:31 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551355/ [5551357] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:05:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551357/ [5551358] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:14:12 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551358/ [5551359] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_latest} fixed-2 mon_election/classic} time: 00:17:26 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551359/ [5551360] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:21:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551360/ [5551361] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/dashboard} time: 01:00:10 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551361/ [5551362] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:05:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551362/ [5551364] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:07:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551364/ [5551365] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:09 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551365/ [5551366] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:25:09 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551366/ [5551367] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551367/ [5551369] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:11:36 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551369/ [5551370] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:19:19 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551370/ [5551371] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 00:34:09 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551371/ [5551372] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/pool-create-delete} time: 00:12:37 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551372/ [5551373] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:30:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551373/ [5551374] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:07 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551374/ [5551376] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:24:56 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551376/ [5551377] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:13:25 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551377/ [5551379] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:07:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551379/ [5551380] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:24:40 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551380/ [5551381] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:15 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551381/ [5551382] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados tasks/rados_api_tests validater/lockdep} time: 00:21:44 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551382/ [5551384] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:09:36 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551384/ [5551385] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:26:21 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551385/ [5551386] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/failover} time: 00:10:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551386/ [5551388] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:18 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551388/ [5551389] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/dashboard} time: 01:00:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551389/ [5551391] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:05:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551391/ [5551392] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:38:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551392/ [5551393] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:44 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551393/ [5551394] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:06:47 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551394/ [5551396] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 00:28:22 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551396/ [5551397] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:26:21 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551397/ [5551398] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:05:40 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551398/ [5551399] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:13:41 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551399/ [5551401] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:08:50 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551401/ [5551402] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:19:16 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551402/ [5551404] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:06:36 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551404/ [5551405] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:12:12 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551405/ [5551406] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:26:11 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551406/ [5551408] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551408/ [5551409] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:23:14 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551409/ [5551410] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:22:43 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551410/ [5551411] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:03:14 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551411/ [5551412] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_latest}} time: 00:12:14 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551412/ [5551413] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:19:41 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551413/ [5551415] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:19 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551415/ [5551417] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:27:32 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551417/ [5551418] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:19:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551418/ [5551419] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:05 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551419/ [5551421] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:09:30 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551421/ [5551422] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:15:49 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551422/ [5551423] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache} time: 00:09:01 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551423/ [5551425] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:27:06 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551425/ [5551426] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:32 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551426/ [5551427] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados tasks/rados_cls_all validater/valgrind} time: 02:04:55 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551427/ [5551429] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:09:49 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551429/ [5551430] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:19:35 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551430/ [5551431] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:08:49 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551431/ [5551433] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:07:54 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551433/ [5551434] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:58:23 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551434/ [5551435] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/misc} time: 00:26:52 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551435/ [5551437] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench} time: 01:26:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551437/ [5551438] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:02:44 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551438/ [5551439] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm_repos} time: 00:04:21 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551439/ [5551440] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect} time: 00:12:44 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551440/ [5551441] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:18:15 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551441/ [5551442] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:07:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551442/ [5551444] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:21:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551444/ [5551445] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:11:03 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551445/ [5551446] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:09:45 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551446/ [5551448] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:15:17 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551448/ [5551449] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:15:28 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551449/ [5551450] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:29:18 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551450/ [5551452] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:15:38 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551452/ [5551453] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:09 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551453/ [5551454] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:22 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551454/ [5551455] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:07:50 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551455/ [5551456] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:22:11 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551456/ [5551458] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:18:19 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551458/ [5551459] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_api_tests} time: 00:28:50 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551459/ [5551460] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:28:04 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551460/ [5551461] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:31:17 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551461/ [5551462] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:19:12 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551462/ [5551464] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:00:00 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551464/ [5551465] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_8}} time: 00:21:59 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551465/ [5551466] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects} time: 00:27:20 info: http://pulpito.ceph.com/kchai-2020-10-23_10:21:03-rados-wip-kefu-testing-2020-10-23-1631-distro-basic-smithi/5551466/