2021-01-14T10:11:36.539 INFO:root:teuthology version: 1.0.0-7ba49335 2021-01-14T10:11:36.541 INFO:teuthology.results:subset: '18/100000' 2021-01-14T10:11:36.541 INFO:teuthology.results:seed: '9064' 2021-01-14T13:02:51.584 INFO:root:teuthology version: 1.0.0-7ba49335 2021-01-14T13:02:51.639 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-01-15T00:56:16.128 INFO:teuthology.results:Tests finished! gathering results... 2021-01-15T00:57:41.141 INFO:scrape:Found 367 jobs 2021-01-15T00:57:50.305 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8998-gcde44442 (cde444426c7255b0ab532dedf6ae7a8e2eef0878) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe1) [0x7f052c0b256b] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x501) [0x55a03494eed1] 2021-01-15T00:57:50.307 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:47:17.938 INFO:teuthology.orchestra.run.smithi028.stdout:} , 2021-01-14T11:47:17.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 , instructed osd(s) 5 to deep-scrub 2021-01-15T00:57:50.307 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:48:40.843 INFO:teuthology.orchestra.run:waiting for 300 , 2021-01-14T11:48:40.872 INFO:tasks.ceph.mgr.x:Stopped , 2021-01-14T11:48:40.873 INFO:teuthology.misc:Shutting down mon daemons... 2021-01-15T00:57:50.308 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:52:57.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2021-01-14T11:52:57.795 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2021-01-14T11:52:57.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-01-15T00:57:50.436 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8998-gcde44442 (cde444426c7255b0ab532dedf6ae7a8e2eef0878) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe1) [0x7f052c0b256b] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x501) [0x55a03494eed1] 2021-01-15T00:57:50.438 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:47:17.938 INFO:teuthology.orchestra.run.smithi028.stdout:} , 2021-01-14T11:47:17.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 , instructed osd(s) 5 to deep-scrub 2021-01-15T00:57:50.438 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:48:40.843 INFO:teuthology.orchestra.run:waiting for 300 , 2021-01-14T11:48:40.872 INFO:tasks.ceph.mgr.x:Stopped , 2021-01-14T11:48:40.873 INFO:teuthology.misc:Shutting down mon daemons... 2021-01-15T00:57:50.439 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:52:57.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2021-01-14T11:52:57.795 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2021-01-14T11:52:57.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-01-15T00:57:50.573 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8998-gcde44442 (cde444426c7255b0ab532dedf6ae7a8e2eef0878) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe1) [0x7f052c0b256b] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x501) [0x55a03494eed1] 2021-01-15T00:57:50.575 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:47:17.938 INFO:teuthology.orchestra.run.smithi028.stdout:} , 2021-01-14T11:47:17.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 , instructed osd(s) 5 to deep-scrub 2021-01-15T00:57:50.576 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:48:40.843 INFO:teuthology.orchestra.run:waiting for 300 , 2021-01-14T11:48:40.872 INFO:tasks.ceph.mgr.x:Stopped , 2021-01-14T11:48:40.873 INFO:teuthology.misc:Shutting down mon daemons... 2021-01-15T00:57:50.576 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:52:57.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2021-01-14T11:52:57.795 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2021-01-14T11:52:57.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-01-15T00:57:50.712 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8998-gcde44442 (cde444426c7255b0ab532dedf6ae7a8e2eef0878) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe1) [0x7f052c0b256b] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x501) [0x55a03494eed1] 2021-01-15T00:57:50.714 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:47:17.938 INFO:teuthology.orchestra.run.smithi028.stdout:} , 2021-01-14T11:47:17.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 , instructed osd(s) 5 to deep-scrub 2021-01-15T00:57:50.715 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:48:40.843 INFO:teuthology.orchestra.run:waiting for 300 , 2021-01-14T11:48:40.872 INFO:tasks.ceph.mgr.x:Stopped , 2021-01-14T11:48:40.873 INFO:teuthology.misc:Shutting down mon daemons... 2021-01-15T00:57:50.715 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:52:57.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2021-01-14T11:52:57.795 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2021-01-14T11:52:57.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-01-15T00:57:50.855 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8998-gcde44442 (cde444426c7255b0ab532dedf6ae7a8e2eef0878) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe1) [0x7f052c0b256b] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x501) [0x55a03494eed1] 2021-01-15T00:57:50.859 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:47:17.938 INFO:teuthology.orchestra.run.smithi028.stdout:} , 2021-01-14T11:47:17.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 , instructed osd(s) 5 to deep-scrub 2021-01-15T00:57:50.860 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:48:40.843 INFO:teuthology.orchestra.run:waiting for 300 , 2021-01-14T11:48:40.872 INFO:tasks.ceph.mgr.x:Stopped , 2021-01-14T11:48:40.873 INFO:teuthology.misc:Shutting down mon daemons... 2021-01-15T00:57:50.861 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:52:57.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2021-01-14T11:52:57.795 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2021-01-14T11:52:57.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-01-15T00:57:50.984 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8998-gcde44442 (cde444426c7255b0ab532dedf6ae7a8e2eef0878) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe1) [0x7f052c0b256b] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x501) [0x55a03494eed1] 2021-01-15T00:57:50.986 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:47:17.938 INFO:teuthology.orchestra.run.smithi028.stdout:} , 2021-01-14T11:47:17.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 , instructed osd(s) 5 to deep-scrub 2021-01-15T00:57:50.987 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:48:40.843 INFO:teuthology.orchestra.run:waiting for 300 , 2021-01-14T11:48:40.872 INFO:tasks.ceph.mgr.x:Stopped , 2021-01-14T11:48:40.873 INFO:teuthology.misc:Shutting down mon daemons... 2021-01-15T00:57:50.987 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:52:57.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2021-01-14T11:52:57.795 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2021-01-14T11:52:57.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-01-15T00:57:51.123 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8998-gcde44442 (cde444426c7255b0ab532dedf6ae7a8e2eef0878) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe1) [0x7f052c0b256b] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x501) [0x55a03494eed1] 2021-01-15T00:57:51.125 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:47:17.938 INFO:teuthology.orchestra.run.smithi028.stdout:} , 2021-01-14T11:47:17.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 , instructed osd(s) 5 to deep-scrub 2021-01-15T00:57:51.126 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:48:40.843 INFO:teuthology.orchestra.run:waiting for 300 , 2021-01-14T11:48:40.872 INFO:tasks.ceph.mgr.x:Stopped , 2021-01-14T11:48:40.873 INFO:teuthology.misc:Shutting down mon daemons... 2021-01-15T00:57:51.126 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:52:57.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2021-01-14T11:52:57.795 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2021-01-14T11:52:57.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-01-15T00:57:51.255 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8998-gcde44442 (cde444426c7255b0ab532dedf6ae7a8e2eef0878) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe1) [0x7f052c0b256b] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x501) [0x55a03494eed1] 2021-01-15T00:57:51.257 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:47:17.938 INFO:teuthology.orchestra.run.smithi028.stdout:} , 2021-01-14T11:47:17.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 , instructed osd(s) 5 to deep-scrub 2021-01-15T00:57:51.258 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:48:40.843 INFO:teuthology.orchestra.run:waiting for 300 , 2021-01-14T11:48:40.872 INFO:tasks.ceph.mgr.x:Stopped , 2021-01-14T11:48:40.873 INFO:teuthology.misc:Shutting down mon daemons... 2021-01-15T00:57:51.258 WARNING:scrape:Ignoring malparsed backtrace: 2021-01-14T11:52:57.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2021-01-14T11:52:57.795 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2021-01-14T11:52:57.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-01-15T00:57:56.304 INFO:scrape:Found 12 distinct failure reasons 2021-01-15T00:57:56.305 INFO:scrape:Dead: b'2021-01-15T00:52:58.226 DEBUG:teuthology.exit:Finished running handlers' 2021-01-15T00:57:56.305 INFO:scrape:5 jobs: ['5785329', '5785067', '5785222', '5785310', '5785332'] 2021-01-15T00:57:56.306 INFO:scrape:suites intersection: [] 2021-01-15T00:57:56.306 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-async-recovery}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'debug/mgr', 'fast/fast', 'fast/normal', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/dashboard/{clusters/{2-node-mgr}', 'rados/objectstore/{backends/objectstore', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/dashboard}', 'thrashers/fastread', 'thrashers/mapgap', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-radosbench}', 'workloads/ec-small-objects-many-deletes}', 'workloads/radosbench-high-concurrency}'] 2021-01-15T00:57:56.307 INFO:scrape: 2021-01-15T00:57:56.307 INFO:scrape:Failure: reached maximum tries (180) after waiting for 180 seconds 2021-01-15T00:57:56.308 INFO:scrape:5 jobs: ['5785061', '5785154', '5785109', '5785205', '5785026'] 2021-01-15T00:57:56.308 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', 'fixed-2', 'rados/cephadm/upgrade/{1-start'] 2021-01-15T00:57:56.308 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{centos_8.0}', 'distro$/{centos_latest}', 'distro$/{rhel_8.0}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2021-01-15T00:57:56.309 INFO:scrape: 2021-01-15T00:57:56.309 INFO:scrape:Failure: Command failed (workunit test cephadm/test_adoption.sh) on smithi153 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' 2021-01-15T00:57:56.310 INFO:scrape:1 jobs: ['5785287'] 2021-01-15T00:57:56.310 INFO:scrape:suites: ['mon_election/classic', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2021-01-15T00:57:56.311 INFO:scrape: 2021-01-15T00:57:56.311 INFO:scrape:Failure: Command failed (workunit test scrub/osd-scrub-snaps.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh' 2021-01-15T00:57:56.311 INFO:scrape:2 jobs: ['5785344', '5785161'] 2021-01-15T00:57:56.312 INFO:scrape:suites intersection: ['workloads/scrub}'] 2021-01-15T00:57:56.312 INFO:scrape:suites union: ['rados/standalone/{mon_election/classic', 'rados/standalone/{mon_election/connectivity', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/scrub}'] 2021-01-15T00:57:56.312 INFO:scrape: 2021-01-15T00:57:56.313 INFO:scrape:Failure: "2021-01-14T10:57:52.251536+0000 osd.1 (osd.1) 3 : cluster [WRN] slow request osd_op(client.4200.0:9245 2.35 2:aceb2a14:::benchmark_data_smithi060_34878_object9244:head [set-alloc-hint object_size 4194304 write_size 4194304,write 0~4194304 in=4194304b] snapc 0=[] ondisk+write+known_if_redirected e15) initiated 2021-01-14T10:57:22.059983+0000 currently delayed" in cluster log 2021-01-15T00:57:56.313 INFO:scrape:1 jobs: ['5785016'] 2021-01-15T00:57:56.314 INFO:scrape:suites: ['mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/balancer', 'supported-random-distro$/{centos_8}}'] 2021-01-15T00:57:56.314 INFO:scrape: 2021-01-15T00:57:56.314 INFO:scrape:Timeout 6h running clone.client.0/qa/workunits/rados/test.sh 2021-01-15T00:57:56.315 INFO:scrape:1 jobs: ['5785029'] 2021-01-15T00:57:56.315 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-zlib', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'validater/valgrind}'] 2021-01-15T00:57:56.316 INFO:scrape: 2021-01-15T00:57:56.316 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi192 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2021-01-15T00:57:56.316 INFO:scrape:2 jobs: ['5785183', '5785053'] 2021-01-15T00:57:56.317 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2021-01-15T00:57:56.317 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2021-01-15T00:57:56.318 INFO:scrape: 2021-01-15T00:57:56.318 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' 2021-01-15T00:57:56.318 INFO:scrape:1 jobs: ['5785153'] 2021-01-15T00:57:56.319 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/many', 'msgr/async-v2only', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'supported-random-distro$/{ubuntu_latest}}'] 2021-01-15T00:57:56.319 INFO:scrape: 2021-01-15T00:57:56.320 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi035 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2021-01-15T00:57:56.320 INFO:scrape:2 jobs: ['5785194', '5785015'] 2021-01-15T00:57:56.320 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2021-01-15T00:57:56.321 INFO:scrape:suites union: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2021-01-15T00:57:56.321 INFO:scrape: 2021-01-15T00:57:56.322 INFO:scrape:Failure: Command failed on smithi145 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:cde444426c7255b0ab532dedf6ae7a8e2eef0878 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9f678fa-5658-11eb-8f90-001a4aab830c -- ceph orch daemon add mon smithi145:172.21.15.145=smithi145' 2021-01-15T00:57:56.322 INFO:scrape:1 jobs: ['5785097'] 2021-01-15T00:57:56.322 INFO:scrape:suites: ['rados/cephadm/smoke-roleless/{distro/rhel_latest', 'start}'] 2021-01-15T00:57:56.323 INFO:scrape: 2021-01-15T00:57:56.323 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi161 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2021-01-15T00:57:56.324 INFO:scrape:2 jobs: ['5785118', '5785316'] 2021-01-15T00:57:56.324 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'msgr-failures/few', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'supported-random-distro$/{ubuntu_latest}'] 2021-01-15T00:57:56.324 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-partial-recovery}', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rados_api_tests}', 'thrashers/careful', 'thrashosds-health', 'workloads/rados_api_tests}'] 2021-01-15T00:57:56.325 INFO:scrape: 2021-01-15T00:57:56.325 INFO:scrape:Crash: Command failed (workunit test cephtool/test.sh) on smithi123 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh' 2021-01-15T00:57:56.326 INFO:scrape:ceph version 16.0.0-8998-gcde44442 (cde444426c7255b0ab532dedf6ae7a8e2eef0878) pacific (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f635a846980] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19c) [0x558cf81efd2c] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x558cf81efeb6] 6: (PG::scrub(unsigned int, ThreadPool::TPHandle&)+0x195) [0x558cf8329a35] 7: (ceph::osd::scheduler::PGScrub::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0x1a) [0x558cf84d749a] 8: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd5) [0x558cf8295215] 9: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x4ac) [0x558cf890797c] 10: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0x558cf890ac30] 11: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f635a83b6db] 12: clone() 2021-01-15T00:57:56.326 INFO:scrape:1 jobs: ['5785096'] 2021-01-15T00:57:56.326 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/many', 'msgr/async-v2only', 'objectstore/bluestore-comp-lz4', 'rados', 'rados/singleton-bluestore/{all/cephtool', 'supported-random-distro$/{ubuntu_latest}}'] 2021-01-15T00:57:56.327 INFO:scrape: 2021-01-15T00:57:56.338 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/ failed: 19 dead: 5 running: 0 waiting: 0 queued: 0 passed: 343 Fail ================================================================= [5785015] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:33:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785015/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785015/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b487a294045f4c6c991f55087efcb119 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi191 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5785016] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:20:11 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785016/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785016/ "2021-01-14T10:57:52.251536+0000 osd.1 (osd.1) 3 : cluster [WRN] slow request osd_op(client.4200.0:9245 2.35 2:aceb2a14:::benchmark_data_smithi060_34878_object9244:head [set-alloc-hint object_size 4194304 write_size 4194304,write 0~4194304 in=4194304b] snapc 0=[] ondisk+write+known_if_redirected e15) initiated 2021-01-14T10:57:22.059983+0000 currently delayed" in cluster log [5785026] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:17:58 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785026/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785026/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4f8cab4115234132b3e0d8db18990dd6 reached maximum tries (180) after waiting for 180 seconds [5785029] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 06:50:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785029/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785029/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ab6bff1af55146da9e5cf34fc6cbc799 Command failed (workunit test rados/test.sh) on smithi120 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5785053] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:16:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785053/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785053/ sentry: https://sentry.ceph.com/organizations/ceph/?query=06184d4fa763439d848050c53fa0abe8 Command failed (workunit test cephadm/test_cephadm.sh) on smithi199 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5785061] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:16:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785061/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785061/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eeb04f7724304da6bf333419da6ff009 reached maximum tries (180) after waiting for 180 seconds [5785096] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:40:33 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785096/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785096/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9ddeed224d5441a2864336a18b2fcaea Command failed (workunit test cephtool/test.sh) on smithi123 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh' [5785097] rados/cephadm/smoke-roleless/{distro/rhel_latest start} ----------------------------------------------------------------- time: 00:07:13 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785097/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785097/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a5ce297250cc43aaad7e2b85800f167e Command failed on smithi145 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:cde444426c7255b0ab532dedf6ae7a8e2eef0878 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9f678fa-5658-11eb-8f90-001a4aab830c -- ceph orch daemon add mon smithi145:172.21.15.145=smithi145' [5785109] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:19:58 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785109/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785109/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fc94763a31bb453fb688091c6dc8f0b5 reached maximum tries (180) after waiting for 180 seconds [5785118] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:29:07 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785118/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785118/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b68fe051dde46dd9fb5b972574f13e4 Command failed (workunit test rados/test.sh) on smithi161 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5785153] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:18:45 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785153/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785153/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [5785154] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:18:51 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785154/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785154/ sentry: https://sentry.ceph.com/organizations/ceph/?query=120c77afaa2b43eebef1cc96c91c1c2a reached maximum tries (180) after waiting for 180 seconds [5785161] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/scrub} ----------------------------------------------------------------- time: 01:20:17 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785161/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785161/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4260abb99181496091f081916f307860 Command failed (workunit test scrub/osd-scrub-snaps.sh) on smithi165 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub- snaps.sh' [5785183] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:16:01 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785183/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785183/ sentry: https://sentry.ceph.com/organizations/ceph/?query=16518819b1d74d73a5ac182507b3d6e5 Command failed (workunit test cephadm/test_cephadm.sh) on smithi192 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5785194] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:33:51 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785194/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785194/ sentry: https://sentry.ceph.com/organizations/ceph/?query=353f3e2f3aeb4231b2943d06982b0486 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi035 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5785205] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:17:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785205/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785205/ sentry: https://sentry.ceph.com/organizations/ceph/?query=99f1e9342027463989d372a7331217e6 reached maximum tries (180) after waiting for 180 seconds [5785287] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:16:24 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785287/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785287/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1a619d0ad16d4d61a5d0c8baa6679c3d Command failed (workunit test cephadm/test_adoption.sh) on smithi153 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5785316] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 02:21:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785316/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785316/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cd5489b04860496ba5dbcb1bf4bb8bf7 Command failed (workunit test rados/test.sh) on smithi122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5785344] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/scrub} ----------------------------------------------------------------- time: 01:23:37 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785344/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785344/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7e92964fb9f346428277a93cbba801ef Command failed (workunit test scrub/osd-scrub-snaps.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cde444426c7255b0ab532dedf6ae7a8e2eef0878 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub- snaps.sh' Dead ================================================================= [5785067] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785067/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785067/ [5785222] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785222/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785222/ [5785310] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/dashboard} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785310/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785310/ [5785329] rados/objectstore/{backends/objectstore supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785329/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785329/ [5785332] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-many-deletes} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785332/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785332/ Pass ================================================================= [5784983] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:42:23 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784983/ [5784984] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:29:44 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784984/ [5784985] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:12:16 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784985/ [5784986] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:18:56 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784986/ [5784987] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784987/ [5784988] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:12 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784988/ [5784989] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/lockdep} time: 00:13:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784989/ [5784990] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784990/ [5784991] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:09 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784991/ [5784992] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:23 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784992/ [5784993] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:08:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784993/ [5784994] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:14:17 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784994/ [5784995] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:41:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784995/ [5784996] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:24:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784996/ [5784997] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:15:02 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784997/ [5784998] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:12:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784998/ [5784999] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:10:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5784999/ [5785000] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:23:35 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785000/ [5785001] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/crash} time: 00:12:21 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785001/ [5785002] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:48:52 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785002/ [5785003] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/dashboard} time: 01:08:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785003/ [5785004] rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_8}} time: 00:18:27 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785004/ [5785005] rados/rest/{mgr-restful supported-random-distro$/{centos_8}} time: 00:19:04 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785005/ [5785006] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:16:13 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785006/ [5785007] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/crush} time: 00:20:21 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785007/ [5785008] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_latest} time: 05:00:18 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785008/ [5785009] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:21:04 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785009/ [5785010] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:30:23 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785010/ [5785011] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_adoption} time: 00:18:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785011/ [5785012] rados/singleton/{all/peer mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:09:14 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785012/ [5785013] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:27:12 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785013/ [5785014] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/many workloads/pool-create-delete} time: 00:19:13 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785014/ [5785017] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{centos_8}} time: 00:13:21 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785017/ [5785018] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:08:18 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785018/ [5785019] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:13:46 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785019/ [5785020] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:20:39 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785020/ [5785021] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:20:56 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785021/ [5785022] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:13:03 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785022/ [5785023] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/dashboard} time: 01:08:34 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785023/ [5785024] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:01 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785024/ [5785025] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:55 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785025/ [5785027] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:13:02 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785027/ [5785028] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:11 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785028/ [5785030] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 00:43:17 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785030/ [5785031] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:20:37 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785031/ [5785032] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:27:07 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785032/ [5785033] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:29:19 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785033/ [5785034] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:07 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785034/ [5785035] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:13:37 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785035/ [5785036] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:13:02 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785036/ [5785037] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:59 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785037/ [5785038] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:45:11 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785038/ [5785039] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:10:19 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785039/ [5785040] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:11 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785040/ [5785041] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:22:35 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785041/ [5785042] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:05:04 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785042/ [5785043] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:28:45 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785043/ [5785044] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:12:12 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785044/ [5785045] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects} time: 00:29:35 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785045/ [5785046] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:47 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785046/ [5785047] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} time: 00:09:08 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785047/ [5785048] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:11 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785048/ [5785049] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:19 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785049/ [5785050] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_5925} time: 00:10:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785050/ [5785051] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785051/ [5785052] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:27:27 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785052/ [5785054] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rados_api_tests} time: 00:37:47 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785054/ [5785055] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:17:29 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785055/ [5785056] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:12:21 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785056/ [5785057] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:05:08 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785057/ [5785058] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:13 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785058/ [5785059] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:13:40 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785059/ [5785060] rados/singleton/{all/random-eio mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:29:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785060/ [5785062] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} time: 00:31:44 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785062/ [5785063] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:54 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785063/ [5785064] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785064/ [5785065] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785065/ [5785066] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/lockdep} time: 00:27:23 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785066/ [5785068] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:22 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785068/ [5785069] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:20:30 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785069/ [5785070] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:08:39 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785070/ [5785071] rados/singleton/{all/rebuild-mondb mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:15:34 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785071/ [5785072] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:11:25 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785072/ [5785073] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785073/ [5785074] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785074/ [5785075] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:08:12 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785075/ [5785076] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects} time: 00:22:02 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785076/ [5785077] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:35:34 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785077/ [5785078] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785078/ [5785079] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:12:24 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785079/ [5785080] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:18:02 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785080/ [5785081] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/misc} time: 00:37:39 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785081/ [5785082] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:28:47 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785082/ [5785083] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:36:53 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785083/ [5785084] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/rados_api_tests} time: 00:37:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785084/ [5785085] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:04 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785085/ [5785086] rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8}} time: 00:08:17 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785086/ [5785087] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm_repos} time: 00:15:06 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785087/ [5785088] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:09:30 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785088/ [5785089] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/scrub_test} time: 00:11:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785089/ [5785090] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785090/ [5785091] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:14:54 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785091/ [5785092] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:40:22 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785092/ [5785093] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:10:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785093/ [5785094] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:41 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785094/ [5785095] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:11:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785095/ [5785098] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:10:18 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785098/ [5785099] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/redirect_promote_tests} time: 00:15:05 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785099/ [5785100] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:47 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785100/ [5785101] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/mon_recovery validater/valgrind} time: 00:27:36 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785101/ [5785102] rados/singleton/{all/test-crash mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:19:36 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785102/ [5785103] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:18:37 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785103/ [5785104] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785104/ [5785105] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:29 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785105/ [5785106] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785106/ [5785107] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:24:49 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785107/ [5785108] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/mon} time: 00:54:45 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785108/ [5785110] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:09:14 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785110/ [5785111] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:25:01 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785111/ [5785112] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:19:29 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785112/ [5785113] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:16:48 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785113/ [5785114] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:30:27 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785114/ [5785115] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect_set_object} time: 00:10:55 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785115/ [5785116] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:29:34 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785116/ [5785117] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_orch_cli} time: 00:16:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785117/ [5785119] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:13:06 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785119/ [5785120] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:44:56 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785120/ [5785121] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 01:17:21 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785121/ [5785122] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:14:51 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785122/ [5785123] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/set-chunks-read} time: 00:16:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785123/ [5785124] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:06 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785124/ [5785125] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:28 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785125/ [5785126] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} time: 02:23:14 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785126/ [5785127] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:16 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785127/ [5785128] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:19:51 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785128/ [5785129] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:25:51 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785129/ [5785130] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:26:58 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785130/ [5785131] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:14:48 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785131/ [5785132] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:09:06 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785132/ [5785133] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:23:09 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785133/ [5785134] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:18:18 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785134/ [5785135] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:21 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785135/ [5785136] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects-balanced} time: 00:19:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785136/ [5785137] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:42:29 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785137/ [5785138] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:27 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785138/ [5785139] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/lockdep} time: 00:34:24 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785139/ [5785140] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785140/ [5785141] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:31:44 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785141/ [5785142] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:30:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785142/ [5785143] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/progress} time: 00:23:27 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785143/ [5785144] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:12:07 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785144/ [5785145] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:18:57 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785145/ [5785146] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:48 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785146/ [5785147] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} time: 00:22:17 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785147/ [5785148] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:23:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785148/ [5785149] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:18 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785149/ [5785150] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:24:41 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785150/ [5785151] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:26:36 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785151/ [5785152] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_adoption} time: 00:18:21 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785152/ [5785155] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:14:34 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785155/ [5785156] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:03 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785156/ [5785157] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:35:51 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785157/ [5785158] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects} time: 00:19:14 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785158/ [5785159] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:11:54 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785159/ [5785160] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:57 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785160/ [5785162] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:18:54 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785162/ [5785163] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:06:01 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785163/ [5785164] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:29:33 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785164/ [5785165] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} time: 00:30:20 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785165/ [5785166] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785166/ [5785167] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{centos_8}} time: 02:07:33 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785167/ [5785168] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:27:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785168/ [5785169] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:12:37 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785169/ [5785170] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:08:22 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785170/ [5785171] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785171/ [5785172] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:05 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785172/ [5785173] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/rados_cls_all validater/valgrind} time: 02:21:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785173/ [5785174] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:48 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785174/ [5785175] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:18:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785175/ [5785176] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:36:01 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785176/ [5785177] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/prometheus} time: 00:11:59 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785177/ [5785178] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:29:12 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785178/ [5785179] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} time: 00:28:02 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785179/ [5785180] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} time: 00:21:01 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785180/ [5785181] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:11:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785181/ [5785182] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:18:24 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785182/ [5785184] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:08:09 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785184/ [5785185] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:27:37 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785185/ [5785186] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:49:05 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785186/ [5785187] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:02 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785187/ [5785188] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/dashboard} time: 01:10:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785188/ [5785189] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:16:15 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785189/ [5785190] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/crush} time: 00:21:08 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785190/ [5785191] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_latest} time: 05:33:23 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785191/ [5785192] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:44:25 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785192/ [5785193] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:28:58 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785193/ [5785195] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:06:24 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785195/ [5785196] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/many workloads/snaps-few-objects} time: 00:35:59 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785196/ [5785197] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:08:44 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785197/ [5785198] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{rhel_8}} time: 00:13:44 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785198/ [5785199] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:19:20 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785199/ [5785200] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:13:24 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785200/ [5785201] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:11:03 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785201/ [5785202] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:09:47 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785202/ [5785203] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed} time: 00:17:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785203/ [5785204] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:13:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785204/ [5785206] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:50 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785206/ [5785207] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:13:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785207/ [5785208] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:11 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785208/ [5785209] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:07:49 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785209/ [5785210] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados tasks/mon_recovery validater/lockdep} time: 00:12:52 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785210/ [5785211] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:18:24 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785211/ [5785212] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:25:41 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785212/ [5785213] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785213/ [5785214] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:15:09 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785214/ [5785215] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:10:28 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785215/ [5785216] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/erasure-code} time: 00:46:14 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785216/ [5785217] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:28:22 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785217/ [5785218] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:12:50 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785218/ [5785219] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:31:36 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785219/ [5785220] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:19:08 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785220/ [5785221] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:09:51 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785221/ [5785223] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm_repos} time: 00:15:08 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785223/ [5785224] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:44:13 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785224/ [5785225] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:12 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785225/ [5785226] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-agent-big} time: 00:17:56 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785226/ [5785227] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:25:48 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785227/ [5785228] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:41 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785228/ [5785229] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:04 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785229/ [5785230] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785230/ [5785231] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:20 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785231/ [5785232] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/one workloads/pool-create-delete} time: 00:20:22 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785232/ [5785233] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:17:01 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785233/ [5785234] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:12:41 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785234/ [5785235] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:12:22 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785235/ [5785236] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-small} time: 00:11:47 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785236/ [5785237] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:25:49 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785237/ [5785238] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/mgr} time: 00:21:21 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785238/ [5785239] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785239/ [5785240] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:40:37 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785240/ [5785241] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:05:39 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785241/ [5785242] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:45:09 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785242/ [5785243] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:41:36 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785243/ [5785244] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:39 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785244/ [5785245] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/mon_recovery validater/valgrind} time: 00:30:48 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785245/ [5785246] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} time: 00:18:14 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785246/ [5785247] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/crash} time: 00:11:54 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785247/ [5785248] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:20 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785248/ [5785249] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:13:17 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785249/ [5785250] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:19 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785250/ [5785251] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:23:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785251/ [5785252] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04_podman} fixed-2 mon_election/connectivity} time: 00:19:05 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785252/ [5785253] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:23:19 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785253/ [5785254] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:46 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785254/ [5785255] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:11:40 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785255/ [5785256] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:38:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785256/ [5785257] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:06 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785257/ [5785258] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:05:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785258/ [5785259] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-balanced} time: 00:20:16 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785259/ [5785260] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_orch_cli} time: 00:16:23 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785260/ [5785261] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:26:06 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785261/ [5785262] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:31 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785262/ [5785263] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:16:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785263/ [5785264] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:19:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785264/ [5785265] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:04:48 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785265/ [5785266] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/misc} time: 00:37:34 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785266/ [5785267] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/rados_5925} time: 00:10:16 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785267/ [5785268] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:12:34 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785268/ [5785269] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/dashboard} time: 01:09:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785269/ [5785270] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/cosbench_64K_write} time: 00:20:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785270/ [5785271] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:29:40 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785271/ [5785272] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:24:49 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785272/ [5785273] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_big} time: 00:41:52 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785273/ [5785274] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:53 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785274/ [5785275] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:12:46 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785275/ [5785276] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:11:35 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785276/ [5785277] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:50:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785277/ [5785278] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:12:05 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785278/ [5785279] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785279/ [5785280] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/lockdep} time: 00:34:58 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785280/ [5785281] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/failover} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785281/ [5785282] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:38:08 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785282/ [5785283] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:28:34 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785283/ [5785284] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:17 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785284/ [5785285] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:53 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785285/ [5785286] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:13:12 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785286/ [5785288] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785288/ [5785289] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_8}} time: 00:33:17 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785289/ [5785290] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:13:45 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785290/ [5785291] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:27:56 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785291/ [5785292] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/mon} time: 00:46:43 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785292/ [5785293] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:11:08 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785293/ [5785294] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/classic} time: 00:18:12 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785294/ [5785295] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:08:48 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785295/ [5785296] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read} time: 00:21:43 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785296/ [5785297] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache} time: 00:08:27 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785297/ [5785298] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:19:40 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785298/ [5785299] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:16:01 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785299/ [5785300] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:08:53 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785300/ [5785301] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:31:18 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785301/ [5785302] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:19:10 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785302/ [5785303] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:28:27 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785303/ [5785304] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_api_tests} time: 00:34:43 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785304/ [5785305] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:19:47 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785305/ [5785306] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:12:47 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785306/ [5785307] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:15:04 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785307/ [5785308] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:28:42 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785308/ [5785309] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:36 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785309/ [5785311] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:41:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785311/ [5785312] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:23:43 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785312/ [5785313] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:17 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785313/ [5785314] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:22:34 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785314/ [5785315] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:15 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785315/ [5785317] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/insights} time: 00:13:50 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785317/ [5785318] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/osd} time: 03:16:26 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785318/ [5785319] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:31 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785319/ [5785320] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/rados_cls_all validater/valgrind} time: 02:22:44 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785320/ [5785321] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:30:11 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785321/ [5785322] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:20:52 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785322/ [5785323] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm} time: 00:23:15 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785323/ [5785324] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:15:39 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785324/ [5785325] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:19:19 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785325/ [5785326] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:12:41 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785326/ [5785327] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:48:18 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785327/ [5785328] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:09:18 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785328/ [5785330] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:24 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785330/ [5785331] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:11:55 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785331/ [5785333] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:15:39 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785333/ [5785334] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:40:54 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785334/ [5785335] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:19:11 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785335/ [5785336] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/connectivity} time: 00:17:38 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785336/ [5785337] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:18:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785337/ [5785338] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:07:54 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785338/ [5785339] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:34:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785339/ [5785340] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:48:11 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785340/ [5785341] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/radosbench} time: 02:07:32 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785341/ [5785342] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/radosbench} time: 01:03:13 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785342/ [5785343] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:30:15 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785343/ [5785345] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:19:00 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785345/ [5785346] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:16 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785346/ [5785347] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:29:51 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785347/ [5785348] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:53 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785348/ [5785349] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2021-01-14_07:01:01-rados-master-distro-basic-smithi/5785349/